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ABSTRACT 

The fairness of testing practices for selection or 
promotion of members of different racial and ethnic groups has been 
questioned. In this study, a search was made of occupations in the 
Federal Government to find those in various agencies with sufficient 
ethnic group representation for acceptable sample size, and for which 
dependable and objective criterion measures might be developed. 
Research conducted over a six -year period focused on the occupations 
of Medical Technician, Cartographic Technician, and Inventory 
Management Specialist. After careful job analyses^ aptitude and 
ability test batteries were selected, several types of criterion 
measures were developed, and background data and job activities 
questionnaires were prepared. Data analysis indicated that tests 
found to be valid for one ethnic group were in gen^-ral valid for the 
other ethnic groups included in the study, across occupations. 
Reg-r.ession equations developed on majority group data appeared to , 
predict almost equally well for minority gYoups, and some instances 
predicted higher criterion scores for minorities. The use of 
supervisors' ratings as a criterion ot job performance in validity 
studies with ethnic sub-groups was called into question by findings^ 
of this study, which disclosed interaction effects ascribed to ethnic 
group membership of rater and ethnic group membership of ratee. 
'(Author/BJG) 
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Foreword 

The study described in this report represents a major cooperative 
effort of the U. S. Civil Service Commission and Educational Testing 
Service, with support from the Ford Foundation. The initial joint 
proposal to the Ford Foundation was motivated by the many questions 
being raised in all sectors as to the fairness of testing practices 
for selection or promotion of members of different racial and ethnic 
groups. Dependable and defensible^ research findings on these questions 
were few, and contradictory results were being reported from study to 
study. ' / 

For the present study, a search was made of occupations in the 
Federal Government to find those in various agencies with sufficient 
ethnic group representation for acceptable sample size, and for which 

e 

dependable and objective criterion measures might be developed. Resean 
was ultimately carried out in three stages over a six-year period, with 
separate studies' in 'depth of the occupation's of Medical Technician, 
Cartographic Technician, and Inventory Management Specialist. 

Each stage began with careful job analysis by researchers to 
determine factors necessary for successful job performance. On the 
basis of the job analyses, aptitude and ability test batteries ware 
selected, several types of criterion measures were developed, and back- 
ground data and job activities questionnaires were prepared. 

The data gathering phases for each occupation were conducted 
after eliciting the cooperation of supervisory staff and job incumbents 
in installations of the government agencies agreeing to participate in 
.the project. All testing was administered by ETS research staff. 



Data were analyzed fojr each occupation by ethnic group so that 
comparisons could be made at every level of statistical treatment. A 
major and important finding of the study is that tests found to be 
valid for one ethnic group are in general valid for the other ethnic 
.groups included in the study, across occupations. Regression equations 
developed on majority group data appeared to predict almost equally 
well for minority groups, and in some instances predicted higher 
criterion scores for minorities. The use of supervisors' ratings as 
a criterion of job performance in validity studies with ethnic sub- 
groups is called into question by findings of this study, which 
disclosed interaction effects ascribed to ethnic group membership of 
rater and ethnic group membership of ratee. 

Findings of this study have implications in particular for employers, 
behavioral scientists, and others concerned with social and public policy 
issues. This research has contributed significantly toward understanding 
a major concern—the fairness of testing. At the same time, it has 
raised questions. It is hoped that some of these questions will receive 
attention^ in the future. 

William A. Gorhara 
Samuel J. Messick 
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" ' Chapter I ^ . 

Introduction 

This is the final report of a study inititited ip late. 1966 to ex- 
• ^ .. / ' ; • ' . ' ' 

plote the relationships. of 'selected background arid ability measures to 
, ' -» * 

various criteria of job perfdnnance for several ethnic groups. The 
project, spanning a six-year period, was a joint effort of the U. S. 
Civil- Service Coimnission arid Educational Testing Service, and was sup— 
por.ted by grants from the Ford Foundation. 

r • ' 

' At the time the proposal to undertake the research was made, 
concern as to the fairness of testing .practices for minority groups 
was widespread.^ Tests were perceived as a barrier to selection an'cl 
promotion of^mLnorities, both in industry and government. The dearth 
of carefully-done research and the sometimes conflicting findings being 
reported motivated the joint undertaking. 

The study was begun after an initial survey of occupations within 
the Federal Government to determine those with sufficient* ethnic sub- 
group-representation. The occupation of Medical Technician (GS-645) 
was selected for the pilot project, in which data were obtained on 

Black and Caucasian job £h<;umbents at specified levels. Experience 

IP 

with^ the pilot study demonstrated the feasibility of conducting such 
resezirch, ajid the findings were of sufficient interest to justify ^ 

funding for study of two additional occupations: Cartographic Techni- 

/ 

cian (GS-1371) and Inventory Management Specialist (GS-2010) . Both of 
these occupations included sufficient numbers of Mexicans-Americans, as 
well as Blacks and Caucasians, for meaningful comparisons between 
ethnic groups. 
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Previous reports .published 

A number reports of the Medical Technician pilot study were 
published: Pike (1969); Flaugher, Campbell, and Pike (1969); Campbell, 
Pike, and Flaugher (1969); Campbell, Pike^ Flaugher, and Mahoney (1970); 
RockX Campbell, and Evans (1970); and Parry and. Mahoney (1970), A 
description of the instrumentation, methodology, and sample obtained in 

r • 

the Cartographic Technician phase can be found in Parry (1971) and for 
the Inventory Management Specialist phase in Crooks and Mahoney (1971) i 
Proceedings of an invitational conference, convened June ,^2"^ 1972, to 
report and critique the principal findings, were also public led (CrOoTcs, 
Ed., 1972). 

Related research ■ * 

Much attention has been directed over the years to differences in 
performance on tests among different ethnic groups. Until comparatively 
recently, however, almost all studies were limited to comparisons of 
mean scores, score ranges, or amount of test overlap, Shuey (1966), 
catalogued some 380 such studies. With great uniformity, these studies 
showed that Black samples scored below corresponding Caucasian samples, 
with a usual difference in .means of about one standard deviation. 

Beg'inning in the 1960's, interest and concern shifted' from test 
score comparisons to test validity and, the fairness of prediction for 
different ethnic groups-. During the last fev; years there has been a^ 
number of studies of the effectiveness of differential prediction in 
industrial or occupational settings. These studies have differed in a 
number of respects: in the type of criterion used; v/hether the tests 
were used for selection, administered experimentally, or administered 
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' » * * ' 

for .some ancillary purpose; the-size of the samples involved; the type^ 
of statistical analysis 'used ; the definition of "test bias," etc. For 
these and other reasons, there has been-* some continuing 'controversy on ' 
the interpretation of results from/^he various studies,. ^ ^ 

As Boehm (1972)- has pointed pyl, the first study of 'this type 



"(Lopez, 1966) produced. the stronges-t indications of differential 

validity yet. obtained . However, -the s^tatistioal^ procedures used^ in 

• • ■ • ^ ^ \ • 

that study were defective^ some*V^spects, which make the .findings ' . \ 

difficult to interpret. Similarly', another early report (Kirkpatrick, 

et 'al., 19^8),.^which included resjsarch on a pumber of jobs and -training ^ 

situations^ produced sey.eral apparent instances of differential validity 

However, the sub-study which produced .th? largest number yof these 

instances was one v;here ethnic group and criterion were cc^n.f dunded, 

again .making interpretation difficult-. • , ' 

• - • • ^ >^ 

Boehm (1972), in reviewing differential validation studies, pointed 

out the difference, apparently firequently Qonfused,^ between what she ^ 

calls "single group validity"> and ^'differential validit>y.'' Differential 

.validity she defines as a situation where "(a) there is a significant 

I V . . ^ , . 

difference between the correlation coefficient of "a selection device and 

a criterion obtained for one ethnic group and the correlation of the 

same device with the same criterion obta^ed for"*the other group, and 

<b) the validity coefficients ara significantly different from^zero for 

one or both groups." Single group^ validity she^defii^es as 4 situation 

where '*a given predictor exhibits validity^ signif itantly different from 

zero for one group only, and there is no significant difference between 

the two validity coefficients." 
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Many of the studies which have reported instances of supposed- 
1 ■ * ' • • » • ^ ■ * ■ 

"differential validity" were in fact examples of "single group valid- 
ity," -^n many* of these instatices, the minority group sample had a 
small N and did not reach the level of statistical significance for 
this reason, even where the validity was in fact larger for the 
minori^ty group than for the majority group. This was true, for 

example, in the studies ^reported by O'Leary, et al, ^ (1970) and Farr, 

' ' 1 
•■ ' . et al. (1971). 

. In abater paper, Boehm C1972, unpublished) has shown that find- 

ings of both^ "single group Validity" and "differential validity" are 

I ' ' ' ' \ ^ 

closely related to jsample size. Most instances of such phenomena 

occur when one or both samples are below 50. 

Ruch (1972) reviewed those studies carried out in a business or 

industrial setting where it was possible to test homogeneity of re- 

gfessioh;. ♦ (A nujaber of studies, including some data from our study of 
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Medical Technicians, were reviewed in both the Bjpehm an analyses.) 
Ruch's cbriclusions were- that__(l) there were fewer differences th<in ^ 
would be; expected by chancy regarding dispersion and^^slope of the 
.regression lines, and (2) there were .more differences than would be 
exp.ected ,by chance "in regression line intercepts", with the intercept 
for the minority 'groups falling below that of the majority group. 

Stanley (1971), reviewing the literature ih the educational 
field/ came to similar conclusions.^ Guinn, Tupes, and Alley (1970), 
^studying prediction of training criteria in ten Air Force technical 
-schools, ^^-^^ found overprediction^f minority performance rather 
than underJprediction. 

. ' 14 
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These findings would indicate (1) that the degree of prediction 

* is as good for minority groups as for majority groups (as was shown 

by. the Boehm review), and (2) that the use of a prediction equation 

basedf;'on a majority sample will tend to overpr edict rather than under 

^ predict the job per^rmance of the^minority group, findings tha*t .are 

' .consistent with results of the study being reported here. 
\ ' • 

In recent" years,, a number of definitions of "test bias" have 

been proposed. 'The one. most often applied is the Cleary (-1568) 

definition. Cleary defines a test as being "culture-fair" for 

population's A*and B when the regression equation based on population 

- A neither systematically over- nor underpredicts the level of 
performance for members of population^^B." This "regression" mod^l 
has been applied in the present study. 

Thorndike (4971) has proposed an alternative definition which 
X^may lead to entirely opposite conclusions as to 'whether a test is 
'*culture-f air ;" He suggests that a test may be judged "culture-fair' 

if the overlap on the crite^rion scores betv;een groups A and B is 

\ 

essentially equivalent to -their overlap on^the predictors. 

- (Additional analyses using the Thorndike approach were carried out 
on data in this study. Theory and results are discussed in Chapter 
XT.) • 

r 

Darlington (1971) and Cole (1972) take slightly different 
approaches to th^ issue, based on the conditional probability that 
"being selected for a job is the same for minority and majority 
^jroups, given satisfactory criterion performance. Linn (1973) dis- 
cusses the major differences in these formula tidhs^ He points out 
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♦ 

that there is \aore than one reasonable definition of test fairness and 
that these definitions are in conflict, so that ,thc choice of definition 
is a matter of weighing competing values • , 
Organization of this report 

This report is in general organized to reflect the order in which 
each phase of the project was carried put, from intensive analy^sis of 
each occupation and factors in job performance, to selection aiid 
development of instrpaents, to data gathering, to basic" analysis of 
data, and then to increasingly sophisticated treatment of the data to 
determine other underlying relationships, patterns, and J-mplicat ions • 

Results from all three occupations are included in each chapter^^ ^ 

A brief description of the content of the chapters follow.: i ^\ 

Chapter II describes the design of the study, identification of \ 

i 

occupations suitable for study, task "analysis of the chree occupations 
chosen, issues in the selection of background and ability measures, 
consi(Jerations in the selection and development of types of criterion 
measures, how the samples were drawn, and how testing was carried out. 
For more detailed descriptions'^f instrumentation development for each 
phase and samples of some of the instruments, see Pike (1969); Parry 
(1971); and Crooks and Mahoney (1971). 

In Chapter III, aptitude and ability measures selected as the " 
predictor batteries for the three occupations are described in relation 
to factors observed in job performance in each occupation. Means and 



Each of these alternative definitions requires an explicit recogni- 
tion of ethnic or o4:her group members^hip. For that reason, use of 
any one of these definitions in emplojmient decisions may run counter 
to the principle stated in the Griggs, vs. Duke Power decision (1971): 
"Congress bas made [job] qualifications the controlling factor so 
^ that race, religion, nationality, and sex become irrelevant." 

ER?C 16 . 



Standard deviations of scores obtained pti the measures are compared by 

ethnic group for each of the three occupations. 

In .Chapter IV, criterion/measures developed for the three occupations 

are described briefly. Means and standard deviations of criterion scores 

arid intercorrelatioi^, of the measures by ethnic group are presented* for 

each^ occupation stiidied. 

Chapter j^;^ea Is with the validity of the aptitude tests for each 

/ ' 

ethnic group as measured by the degree of relationship of aptitude test 

scores with the various criteria of job performance for each occupation 

studies . . \ ' 

/ft . 
A Chapter VI explores differences in the linear relationships (regres- 

S/ion JLines) between test scores and criterion measures by ethnic gr.oup by 

' " ' ' 

//occupation. . 

In Chapter VII, results 'are described of stepwise multiple .regression- 
analyses, in which a best set of predictors was selected by ethnic group 
for each criterion Measure for each occupation. Results of cross-ethnic 
cross-validation,' in which, prediction equations derived for each etijnic 
group were then applied to data for the other ethnic samples'; are also 
presented. • " > ' 

In Chapter VIII, the possible biasing effects of elhnic group member- 
ship of raters (supervisor^ in interaction with ethnic group membership 
of ratees (job incumbents) are described for the three occupations studied. 

Chapter IX presents factor analyses of predictor and criterion vari- 
ables to determine whjether diff-erential patterns of abilities may be 
observed from one ethnic group to another. Sele.cted background variables 
were added by extension. 
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Chapter X presents comparisons of .background and task variables by 
ethnic group b ccupation. Relationships of selected background vari- 
ables to measures of aptitude and job performance are compared by ethnic 
group. Results of task analyses carried out to determine whether there 
were variations in the type of work done by members of different ethnic 
groups are presented. 

Chapter XI contains a theoretical ^scussion of the Cleary (1968) 
and Thorndike (1971) definitions of "culture-fair" tests, and a re- 
analysis of ethnic sample data from this study using a modified 

I 

Thorndike approach. 

Chapter XII presents the major findings in the study and their 
implications. Recommendations for future approaches to some of the un- 
resolved problems are given*. 
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' " Chapter II 
Description and Design of the Study: Three Occupations 

This chapter deals with identification of occupations suitable for 
study, analysis of the three occupations chosen, issues in the selection 
of background ability measures judged most likely to relate to job^ 
performance in thnse occupations, considerations in the selection and 
development of the* three types of criterion measures used, and how the 
samples were drawn and testing implemented* For detailed descriptions 
of the instrumentation development for each^ phase, see Pike (1969); 
Parry (1971) ; and' Crooks and Mahoney (1971)* 
A. Preliminary Study of Occupations 

The first phase of the program was a feasibility study to identify 
the range of technical and administrative issues. A critical first 
step .was to determine whether there were sufficient numbers of minority 
group members in selected occupations to support a full-scale study. 
Some of the stipulations in identifying occupations suitable for ;study 
were: the occupational groups should be ^recognizable by occupational 
^classification series or specialty, the incumbents must include sizeable 
numbers of more than one identifiable ethnic group, written tests miist 
be an acceptable method for measuring job qualifications, it should be 
•possible to obtain objective measures of job performance, and the 
"Encumbents should have the samd basic skills in common acrosq^ instal- 
lations or specialties. An added qualification was that those in an 
occupation suitable* for study should be employed in a number of 
geographic* locations, and include persons with a diversity of back- 
grounds.,. 

-9- 
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After securing occupational data on minority employment in 

selected professional, technical and clerical jobs from nine agencies 

within the Federal government-; it appeared that selecting appropriate 

jobs for study would be no problem at the lower grade levels (GS 1 

through 5). At the higher levels, only a few jobs were a possibility. 

The process of gaining entry and managing negotiations with the 

^ various agencies under consideration was handled by United States Civil 

Service Commission '(USCSC) representatives. IJhe purpose of the study 

was discussed with representatives of minority groups, stresiing the 

n 

intentions of the research and security of individual data. 

Representatives of USCSC.also met with representatives of five 
Federal employees' .unions and informed them of the project; American 
Federation of Government Employees, Government Employees Council, 
National Alliance of Postal and Federal Employees, National Association 
of Internal Revenue Employees, and National Federation of Federal 
Employees. They were asked to make known to their locals that they 
had been briefed, and, approved members' participation in the data 
collection. \ * 

Medical Technicians 

The occupation selected for the feasibility study was Medical 
Technician (GS-645). It was chosen because Blacks and Caucasians were 
represented in this occupation in sufficient numbers for study, the 
■ tasks which the incumbent£_£e^^^ visible 
and objective outcomes that would lend themselves to evaluation, and 
a large propor,tion of the entire population was .employed by a single 
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U.-S. Departments of the Treasury, Commerce, Army, Air Force, Social 
Security, Veterans Administration, Agriculture, Labor, and Defense 
Supply Agency 

20 
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government agency, the Veterans Administration, at various geographic 
locations. One particular advantage of studying, this occupation was 
that written test scores were not a part of the basis-^fpr/liiring the 
majority of technicians. Thus, there should be I'kss restriction of 
range due to preselection by test scores* Such preselection would 
tend to lower expected relationships between test scores and job 
performance criteria. 

A count was made by the Veterans Administration of the. number of 
all personnel in the Medical Technician occupation (GS-645) at each 
installation by grade level, specialty (bacteriolo^T^hematology , 
histopathology, etc.), and ethnic group; The Veterans Administration 
asked supervisors to identify employees individually by ethnic group, 
since Federal regulations prohibit ethnic identification on personnel 
records. 

During Autumn of 1967, five ETS staff members visited nine 

Veterans Administration Hospital laboratories in several parts of 
2 

the country . Before each hospital was visited, a letter was sent 
from the Veterans Administration Washington headquarters to the 
hospital *s director, explaining the purpose of the research program 
and the necessity for the visit, ''naming ETS researchers who would be 
comjjig^. and. .briefly descriMng"wha they proposed to accomplish while 
they were there and what the later phases of the study would entail. 
The support and assistance of the director and the staff were ^ 
encouraged. 

In these preliminary visits, every effort was made to gain as 

^San Francisco and Livermore, Calif., Jackson, Miss., Lyons, N. J., 
Manhattan, N. Y., Washington, D. C, Dayton, 0., Wilkes-Barre, Pa., 
and Beckley, W. Va. 

21 
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thorough an understanding of the occupation of Medical Technician as 
possible, including preliminary education, training, skills, and 
abilities needed; kinds of tasks involved; and aspects of the job which 
were thought most important for successful performance^ Hospitals of 
various sizes with personnel and facilities for treating diverse types , 
of disability were selected in different geograp|iic locations • Admin- 
istrative and personnel officers were interviewed • A brief hi'story of 
each hospital was obtained • Hiring patterns and area "economic condi- 
tions were examined • Medical Technicians at representative GS-levels 
and their supervisors were interviewed and observed at work« Super- ' 
visors often were medical technologists or M^D^s specializing in 
pathology, and from them insights relating to peripheral problems in 
the. laboratories were gained. Particular emphasis was placed on fact- 
finding for a thorough job analysis, based on intervi,fiws.,-^observations, 
and Civil Service. Position Descriptions. Incumbents were as^ed how 
they spent their time each day— rwHat was important, wh&t-.unimportant?- 



necessary? Personnel files were examined to^eterrcine ^the kinds of 
personal history variables that ^hould be^/cumented in the full 
study. A collection of standard^recor^^forms, written instructions, 
and specifications routinely used in the laboratory was made.. The 
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^ How did they learn to oper^ate the equipment thej^sed? How routine 

was its operation?- How highly automated? Wefe they trained \n^dipth 
in one specialty or in several? Did they e^er attend serair^^rs or 
workshops where new equipment or techniques ^ere demonstrated? What * 

constituted a serious error, and what were it^ consequences?. What 

M \ 
qualifications and personal characteristics dj|d they feel were 



published literature was reviewed for any past research on this occuy 
pation. After the selection and design of the actual assessment 1 
materials had begun, several return visits were made to the hospital 
laboratories in order to verify or correct some impressions and to 
obtain further jpb information. 

The feasibility study demonstrated the enormous technical and 
logistic ^difficulties of conducting such research. The value of 
thorough job analysis in planning the instrumentation for such a 
study was clearly confirmed. Since performance on the job is multi- 
dimensional and complex, it is necessary to acquire knowledge of the 
job in depth before selecting aptitude tests, constructing rating 
scales, or developing a work sample. In addition, information about 
the baclcground of job incumbents is necessary to permit construction 
of a comprehensive personal history questionnaire. 

Cartographic Technicians 

The occupation selected for s^:udy in the second phase of the 
project was Cartographic Technician (GS-1371). One of the major 
reasons this occupation was chosen was that job incipben^^ — 
large numbers of both Mexican-Americans and Blacks. Thus, relation- 
ships of predictors to job performance could be* compared for an 
additional ethnic group. It was also an important objective to see 
whether the findings for the Medical Technicians would be replicated 
for the Cartographic Technicians. 

Most Cartographic Technicians are concentrated in three govern- 
ment agencies: U. S. Department of the Army - Corps of Engineers 
and Topographic Command, U. S. Department ot Commerce - Coast & 
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Geodetic Survey and Bureau of the Census, and S. Department of the 

Interior - Geological Survey • Early in 1968, six ETS staff members 

3 

began visiting seven government mapping installations • The same 
general pre-visitation procedures were followed as for the Medical ^ 
Technicians. A count was obtained through the USCSC of incumbents by 
grade level, specialty, and ethnic group. Particular care was exer- 
cised in informing heads of installations of impending visits so that 
necessary advance preparations were assured, including letters to 
supervisory personnel and clearance with union officials. 

One obstacle to the study of this occupation concerned government 
security regulations. Since much military mapping was currently in 
progress, ETS researchers had to be granted security clearance by the 
USCSC. before they were permitted to visit most pf the mapping instal- 
lations. Often it was impossible to interview and observe the 
Cartographic Technicians on the job because they wfere mapping 
restricted areas or using classified equipment. Researchers were 
assured, however, that_the same basic skills and abilities were needed 
to perform the classified tasks, .^and that job analyses based on 
observations of unclassified projects would be adequate. 

After the preliminary visits had been completed, the following 
conclusions were reached: . 

(1) the secure nature of the' work at some installations would 
not preclude the inclusion in the study of the technicians working 
on classified assignments. » 

(2) the varied nature of the work done by Cartographic 

% 

3 

Arlington, Va. , Rockville-Silver Spring and Bethesda, Md., Norfolk, 
Va., Providence, R. I., San Antonio, Tex., and Detroit, Mich. 
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Technicians would require more than one work sample task. 

(3) practical considerations would require excluding Cartographic 
Technicians from the study at installations employing less that 20 
incumb en t s . ^ ^ 

(4) the existence of a substantial number of Mexican-Americans 

in GS-1371 positions would enable an investigation into the test per- 

\ 

formanoe of a second minori-ty group. 

(5) Cartogra^ic Technicians employed by the U. S. Department of 
the Interior - Geological Survey would not be included because of 
insufficient minority group representation. 

Inventory Management Specialists W 
The third and final phase of the study concerned the occupation 
of Inventory Management Specialist (GS-2010). These specialists are 
primarily employed in defense agencies, in a limited number of 
locations, and include large numbers of Blacks and Mexican-Americans. 
This .ciassificcttion-^^spamTed a higher GS-level than the other two 
studiei, and appeared. to require a different set of skills and 

J 

abilidies. 

In the Spring of 1968, preliminary visits were made to two 

F 

Depar^tment of Defense agencies in Philadelphia. Approval of funding 
for this and the other phases of the study was not received until 



late 1 1969. In July, 1970, members of the ETS research staff began 
visiting installations of four agencies of the Dep^ment of Defense: 
Defense Personnel Support Center and Defense Industrial Supply Center, 
infsiallatiuus of the Defense Supply Agency in Philadelphia, Navy 
Aviation Supply Office in Philadelphia, Army Tank-Automotive Command 
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in Detroit, and San Antonio Air Materiel Area (Air Force) ♦ Agair^, 
coptaots were made with the agencies by the Civil Service Commission 
through Washington^ headquarters. An effort" w«-\s made, to meet with 
military administrative heads' of the installations, as n^ell as 
personnel officers and civilian representatives of higher management 
levels, to introduce the purpose of tlie research and to emphasize the 
importance of getting their people to cooperate. On-the-job training 
instructors were interviewed and their lesson materials examined* 

A primary area of concern was the route by which individuals 
entered these jobs. Extensive questioning revealed that incumbents 
were Vhosen in a number of different ways. ^ Many had been screened on 
a written test (Federal Service Entrance E::amination) for entry into 
the career development training program at GS level 5, some had quali- 
fied on the basis of experience at lower levels, and others had been 
transferred or were newly hired from another occupation. 

Because -of the diversity of items managed, special emphasis was 
placed during the interviews on determining the areas of possible 
variation among and between jobs, general activities, perceived as 
common across jobs and agencies and common problem areas, arid areas 

where differences were perceived across jobs or agencies. Again, 

/ 

samples of forms, written procedures, job descriptions, and communi- 
catioris of various kinds were gathered. A number of Inventory 
Management Specialists interviewed were asked to keep a Daily Activity 
Log for five days, and to record, -^at half-hour intervals, activities 
and interactions with others in the process of accomplishing their 
work. 




B* Selection of Aptitude Tests (Predictors) • 

oin selecting aptitude tests for each of the three occupatipns 
studied, researchers utilized their knowledge and familiarity with the 

jdb, gained from interviews and work observations. Considerations 

I * ... 

were finding or developing tests that measured specific abilities ob- 
served as needed for the job, tests of 'general abilities which had been 

* 

found in previous research to relate to job success, and. test s-which, 

were purported to measure other possibly job-related ability factors. 

One important basis for, selection was^ time required to take the tests, 
^ ' : ' ^ ^ i - 

since time spent away from the subjects* regular jobs was. a necessary ^ 

consideration, especially for the Medical Technicians. Ease of group ' 
administration and face -validity were also felt to be important. Other" 
considerat^ians were the availability of tests with two separately timed 
halves, to facilitate reliability estimation, and the need foi; speeded 
versus unspeeded tests.. 

It was decided to use available tests rather than to develop new 
ones.. After considering a number of alternatives, the French, et al . , 

"^Ki t of , Reference Tests for Cognitive Factors became the source of' the 

> — z 

majority of tests in the predictor batteries for all three occupations. 
These tests have short time limits, are divided into separately timed 
halves, have known factorial content,, and can be group-administered 
easily. Since these tests were designed for research i^rposes, the ^ 
. subjects used in this study probably would not have taken them previ- • 
ously. Four U. S. Civil Service Commission tests were used. 'For the 
Cartographic Technicians, the Coordination Test of the Flanagan Indui;- 
trial Tests series was also selected. 



In the next chapter, the aptitude tests will be described as they 
relate to specific aspects of each occupation studied. 

C. Selection and Development of Measures of Job Performance (Criteria) 

For the first two occupations studied (Medical Technician and 
Cartographic Technician), three types of job performance measures were 
developed. Criterion Rating Scales, a Work Sample, and a Job Knowleidge 
Test. Since preliminary study of the third occupation (Inventory 
Management Specialist) yielded wide variations in specific knowledge 
needed and procedures followed wi*thin and among installations, a 
decision was made not to develop a Job Knowledge Test, But to attempt 
to measure the variance in performance attributable to job knowledge 
in che Work Sample and Criterion Rating Scales. 
Rating Scales 

During the preliminary interviews and observations of each occu- 
pation, opinions were solicited concerning job requirements and the 
qualities that were needed for successful job performance. Informa- 
tion obtained by the various interviewers was then pooled, and a list 
of job factors that appeared to be important was identified.. 

Additional visits were made, with emphasis on obtaining a broader 
I 

sampling of activities, confirming or correcting original impressions, 
and evaluating the degree of similarity within* job classification 
across laboratories, installations, or agencies. Interviewees were 
asked to evaluate ^nd comment on the job factors previously identified 
as being important. A tentative list of ^factors was decided on as a" 
basis for developing a set of rating scales, and further Judgments 
concerning their suitability were made. Some were combined because 

28. 
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the definitions were ovetlapping, or were eliminated because the factors 
^ere judged to be of lesser importance. Descriptions of the job factors 
*includfed in the sets for Lll three occupations are given in Chapter IV. 

For all three occupations stujiied, the anchored rating scale format 
(Smith and Kendall, 1963), was used, in which scales are anchored by ex- 
amples of specific behavior. Although some of the same characteristics 
were felt to be important for all occupations being studied, all 'scales 
and' behavioral examples were defined in terms of the specific job under 
investigation. 

Detailed instructions were given at the beginning cf the booklet 
of rating scales, and a list of incumbents to be rated, along with 
identification numbers assigned to them, was included at the end of the 
booklet. The Medical Technicians were rated by co-workers and tvo super- 
visors whenever possible. For the Cartographic Technicians and Inventory 
Management Specialists, only supervisors' ratings were obtained.. Peer 
ratings were dropped because of time pressure, impracticality, and 
possible adverse reaction. For the latter two occupations, the super- 
visor was asked' to indicate how long each person he rated had been under 

his' jurisdiction. 

For all three occupations, raters were also asked to evaluate the 
performance of a fictitious incumbent whose characteristic behavior was 
described on the page opposite each scale. This standard stimulus was 
used in an attempt to disclose stylistic variations or bias in rating 
behavior. For discussion see Appendix to Chapter II.. 

Whenever possible, the raters met in groups, with the researcher 
leading them through the rating procedure. Ratings were made for all 
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ratees by their supervisors on one scale at a time. The purpose was 
to focus on the definitions of the scales, and to emphasize the 
experimental and confidential nature of the study and the importance 
of valid ratings. 

For the second and third occupations studied, each supervisor, 
after completing the ratings, was asked to indicate on a separate 
form how important ^he characteristic described on each of the scales 
was to overall job performance from "Not Important - Irrelevant to 
Proper Job Performance" at the low end, to "Important - Contributes 
to Proper 'Job Performance" at the midpoint, and "Very Important - 
Essential to Proper Job Performance" at the high end of the scale. 

Job Knowledge Test i 

Job. knowledge is one aspect of job performance that can be 
measured directly. To construct a valid test of job knowledge, 
collaboration is necessary between persons who know the job and 
measurement experts, in writing test items that will effectively 
sample the technical knowledge required for successful job perfor- 
mance. Preliminary testing of the items on a sample of job 
incumbents, analysis of these data, and selection of the items for 
the final form of the test which best meet measurement objectives 
are additional required steps. 

^ In search of likely' sources, no appropriate test was found 
that co'uld be administered to Medical Technicians in the first 
occupation studied. However, through informal contacts it was 
discovered that one medical laboratory training school had developed 
a large number of test items over a period of years. A total of 
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584 items from the Allegheny General Hospital Medic|il Laboratory Assis- 

f 

tant Training Program in Pittsburgh, Pennsylvania, fw^ made available. 

After reviews -by two supervisors of Medical Techni^'cians, one from a VA 

f 

hospital and one from a civilian hospital, 147 .multiple choice items 

were selected as most likely to differentiate among technicians on the 

/ 

dimension of job knowledge. These items were pretested and a selectioi 
was made for the final form, as described later/in this chapter. 

For the Cartographic Technicians, a test of basic job knowledge 

was, developed especially for this project. EXS researchers worked 

I 

with three cartographers f rom U. S. Army '^OPOiCOM, one from each of 
the major divisions where Cartographic Technicians are employed: 
cartography, photogrammetry , and triangulat^-on. All of these carto- 
graphers had knowledge of the work done injthe other divisions, and 
could therefore woi;k together effectively as a team in writing test 
items. Evety effort was made to Tiave the test content pertain to 
che types of^i^wledge necessary for successful job performance. 
Information collected from the task list of the Preliminary Back- 
ground Questionnaire was helpful in accomplishing this. 

About 200 multiple choice items were written. The majority of 
questions covered the work in the cartographic area, which is the 
entry-level for this job. The largest number of technicians in the 
sample to be tested worked in this specialty. About ualf as many 
photogrammetric items and about 10 triangulation items were written. 
ETS staff members edited all the items, randomized the alternatives, 
and arranged the items into subject matter categories within the 
three major areas. 
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The items were then reviewed by three additional cartographers 
at S* Army TOPOCOM, The accuracy of the stem, keyed answer, and 
inaccuracy of the distractors were checked for each item. Any items 
lelt to be ambiguous were either clarified or omitted, and those 
where the correct answer could not be agreed upon were eliminated, 
About^lS new items were written in several areas not adequately 
covered. The resulting 165 'items;'were put into two ^ test 'forms and 
the items divided by subcategory within the major areas of carto- 
graphy, photogrammetry, and triangulation, ^ A*^description of the 
pretesting and development of th'e final form appears In Section F 
of this chapter, , . ' 

Work Samples 

Perhaps the most difficult aspect of the project was the construe- 

tion of an adequate work sample for each of the three occupations, to 

/ , ■ * 

ensilre that the measures were unbiased insofar as possible, that the 
method of administration was fair to all, and that ,the tasks selected 
closely approximated what was being done on the job. Although the 
^development of work samples is costly and administration is time- 
consuming for both the administrator and the persons taking the test, 
the Advisory Committee felt that inclusion of such criterion measures . 
was of vital importance in- view of the shortcomings of criteria usually 
used in validity studies, ^ 

During the preliminary job analyses, it became, evident that many 
deterrents to validity and reliability stood in the way of developing 
acceptable measuring instruments, A given job cla siificati on is 
often divided into specialties. Although emphasis is placed on 
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personnel being able to work in more than one specialty, this isi not 
always the case. Especially in the larger installations^ many incum- 
bents have tended to develop more skills in one area than in another. 
Also, a wide variety of tasks within a^ given specialty make a 
representative sampling extremely difficult. 

Some of the problems that had to be considered^ were: How 
representative of the total job is the particular task chosen for 
^the\work sample? Is the difficulty level appropriate? Is the task 
performed the same way at all installations? How much variation is 
there in equipment needed to perform the task?' Does the final product 
adequately reflect the skills and' knowledge employed? \^^hat is the 
optimum testing tiine that can be allocated to this test? 

In developing the work samples for the Medical Technicians, the 

A 

following guidelines were employed: 

(1) limit maximum testing time to 2 to 3 hours. 

(2) use standardi-zed specimens having known values, against 
which to evaluate laboratory test result^. » 

(3) select laboratory tests that tap a variety of task components 
and are relevant for technicians in most specialties. 

(4) minimize variance due to between-laboratory differences such ' 
as available equipment and favored , procedures. 

The work sample selected for Medical Technicians was a laboratory 
fjimulation in which technicians were required to ^conduct teats (an 
alkaline phosphatase and a differential cell count) on standardized 
specimens.' These were decided on after extensive consultation with 
medical laboratory personnel in Veterans Administration and civilian 
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hospitals, and with representatives of pharmaceutical companies en- 
gaged in providing quality control materials and services to their 
laboratories. 

The difficulties encountered in the attempt to. collect adequate" 
work sample data for the Medical Technicians proved to. be greater 
then anticipated. Both' tasks were pretested before, the main admin- 
istration. The pretest appeared to proceed smoothly, and a decision 
was made to go ahead. However, during the main administration, many 
unforeseen problems in preparation and implementation arose. The 
sample of incumbents was very small (N = 34-40) and was chosen from 
two specialties, chemistry and hematology. Test administrators met 
with some initial resistance due to lack of information regarding 
the purpose of the research, but when the intent was clarified sub- 
jects were cooperative. Working conditions were very crowded.. It 
was necessary to use some reagents and equipment from the hospitals, 
causing problems in standardization of specimens. There was too much 
variance in quality of reagents, equipment, and general physical 
setting of the laboratories. 

The tasks proved to be new, unrepresentative, and too difficult 
for the majority of subjects.- The less competent took a long time 
to complete the tasks, and asked many questions. It was felt that 
the results were unreliable and should not be included in analysis 
of other data. The experience was not in vain. Much was learned 
that enabled the researchers to design work s^'^oles for the other 
' occupations studied which would fulfill their criterion role 
reliably. 
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In devising the work sample tasks for the Cartographic Techni- 
cians, the aid of the directors of the three U. S. Army TOPOCOM 
divisions for each mapping specialty was enlisted. They and members 
of their staff met with ETS representatives and discussed possible 
criteria and the kinds of work assignments that might be used. A 
number of tasks was considered, from which three were selected: a 
logical contouring problem, a "pull-up" from an aerial photograph, 
and a geometric restitution task. These were chosen because it was 
felt that they closely approximated what Cartographic Technicians 
actually do on the job, were familiar to all technicians, could be 
administered in a standard fashion, and had characteristics that 
would allow them to be reliably scored. Also, to assure subjects 
that their own supervisors would not see their work, the selected 
tasks could be administered away from their work locations and scored 
without using U. S. Army TOPOCOM staff. Each of the three tasks 
selected required about an hour to finish. 

Because of the complexity of the tasks, a comprehensive scoring 
method was necessary. A brief description of each task and criteria 
for scoring follow: ^• 

Logical contouring - given the drainage pattern and spot eleva- 
tions on a plastic master sheet, technicians were asked to compile 
contour lines at 20-foot intervals on a plastic overlay. Each of the 
three hilltops depicted on the map was scored for size, location, and 
number of contour lines. Correct placement of contour lines at the 
upper right and lower left-hand corners of the area shown was credit- 
ed. A contour line Indicating a 500-foot elevation had to be 



# -26- 

identified and run continuously through a number of specific points on 
the -map. Contour lines had to be evenly spaced throughout • 

Pull-up - subjects were asked to extract the drainage -system and 
cultural detail from*ohe 'of three vertical aerial photographs, using 
a hand stereoscope. This was also compiled on a plastic overlay* 
Scores were based on prop^'c placement and identification of power 
lines, roads, railrcadsj-^ponds, town3 and buildings, intersections, 
and drainage patterns. ^ . v 

^ Geometric restitution - this task required the technicians to 
reconstitute information given on an oblique photograph to a vertical 
format, using a blank plastic worksheet and a small triangle to compile 
the new details. This was scored"by crediting corner ticlcs, parallel- 
ness and accuracy of placement of roads and railroads, and placement, 
shape, and size of buildings. 

Scorers were given extensive preliminary training in scoring 
methods for the Work Sample tasks. Fifty protocols from each of the 
tasks were independently evaluated by two scorers as a reliability 
check. 

During the preliminary study of the job of Inventory Management 
Specialist, it was found that the general activities iti inventory 
management at the different job levels and the knowledges and abilities 
needed to do the work apply across agencies, despite the fact that 
wide variations occur in specific procedures. Although there was a 
number of constraints associated with the selection of a work sample 
which would be appropriate across installations, a common communication 

system was found to exist which had been developed to expedite 

^ ^^^^ 
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procurement, supply, and control of inventory items witlVin and across 
agencies of the Department of Defense. This'^s called the Military 
Standard Requisitioning and Issue Procedure (MILSTRiP). 

It was decided to develop a work sample utilizing the "in-basket" 
technique (Frederiksen, 1957; Hemphill, Griffiths, & Frederiksen, 1962; 
Crooks, 1968) to simulate the general and procedural framework of inven- 
tory management and the role of an inventory manager. In such a test, 
the person taking the test is provided with instructions and selected 
background information. He is asked to work on a set of problems in a 

z 

.specific tfrae period as if he were indeed in the job described in the 
instructions; that is, actually to make and communicate decisions, 
to give directions, to respond to requests from others, to ask for 
information needed to solve problems, to consider priorities in plan- 
nifig and scheduling work, etc. All action taken or planned must be 
written down in the form of communications to .others or notes to self. 

In developing the Inventory Management Specialist Work Sample, it 
was decided that the setting must be unique; that is, different from • 
any existing agency. An hypothetical agency, U. S. Aero"^pace Research 
and Development Administration, was created, as of July 1, 1972. The 
new Inventory Management Specillist working for this agency was to 
manage^ items concerned with life support of crews of two space stations 
(food, clothing, medical items, spare parts for the environmental and 
waste disposal system,^ and some ^ items with a repair cycle). The set 
of 82 items to be managed was somewhat representative of the range of 
items managed across present agencies, although specifically described 
for space station use. Forms and stationery were adapted from samples 

I 
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obtained on visits tcfc installations. By establishing a new and differ- 
ent setting, it 'was possible to extract from the extensive MILSTRIP 
coding and other procedures only those elements necessary to work on 
the set of 19^ problems provided. Some of the problems were related in 
content, and information in iJoth background material and other problems 
cpuld be utilized in taking action. An Action Form was developed which 
subjects were asked, to complete at the end of working time on the 
problems. On this form, they were to describe briefly the action taken 
and reastjns for their action. 

The Work Samples obtained in the pretpsting were used in develop- 
ing the scoring procedure, following an approach described by Crooks 
(1968, 1972). Scoring dimensions are based on administrative skills 
and abilities disclosed as important to performance in early study of 
the inventory management job, and to relevant scbring dimensions used 
for otner "in-basket" exercises. The ten dimensions fall into five 
general categories: (1) to describe what kind of action was taken, 
(2)- to describe how action was taken, (3) the amount of work accom- 
plished (productivity), (4) the quality of the action taken 
(appropriateness), and (5) overall effectiveness of performance in 
the assumed role. 

D. Measurement or Description of Background Variables 

A necessary step in designing the instrumentation was the inclu- 
sion of a personal history questionnaire to determine the composition 
of the work force. Since it was expected that there would be wide 
variation in background characteristics of subgroups, some differences 
in test and job performance for a given Civil Service grade level in 
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a iJarticular occupation might appear to be related to ethnic group 

meirbership when they actually reflected dissimilarities in education 

and background. Unless the background characteristics of majority 

and minority subgroups were analyzed, a study of the job in question. ■ 

would not yiel^ interpretable results. 

j Subject matter for the questionnaires was gathered by many 

personal interviews with incumbents, supervisors and administrative 

staff members, and, f or^ the Medical Technician study, a search of 

hospital personnel records. Form and content of the questionnaires 

were also influenced by a survey of Civil Service Personnel Research 

Questionnaires, the Student Information Blank from Project TALENT 

(Flaiagan, 1964), Glennon and Albright's Catalog of Life History Items 

(1966), and similar sources* 

\ Questionnaires were designed to furnish what was judged to be the 

most 'pertinent background information: age, sex, length of government 

service, socioeconomic level, amount of education ^r^d where received, 
1 * 

natuije and length of work experience, job. held immediately prior to 

current position classification, S. Civil Service grade level at 

entry into occupation, and whether test performance^ had been a basis 

* for Jbtaining the job. Also covered were the comparative roles of 

experience and formal training as determiners of Civil Service grade 

level achieved, and the time taken to achieve it. Many of the 

questionnaire items were identical for all three occupations studied. 

* 

Questions dealing with type of work done, however, were different 
for each job. 

The questionnaire developed for the feasibility study was 
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deliberately overinclusive, to reduce the likelihood of overlooking 
any- background variables which might prove relevant in moderating the 
prediction of job success* With the addition of a third ethnic group 
in the second and third occupations studied, a number of questions 
relating to English language facility were also introduced ♦ 

A brief preliminary questionnaire was mailed to about 1500 
Cartographic Technicians in August, 1968 ♦ This number included techni- 
cians in !!♦ S» Army TOPOCOM, Coast & Geodetic Survey, Bureau of the 
Census, and Army Corps of Engineer s* Results frpm these preliminary 
, questionnaires played a large part in the decision to select only 
those technicians working at U. S* Army TOPOCOM for the main study 
Subjects' from all three occupations were given the Personal History 
Questionnaire at the time of the main test administration* 
E* Measurement or Description of Occupation by. Job Elements or 

Task List 

For all three occupations, incumbents were asked to complete a 
task information checklist as an addendum to the Personal History 
Questionnaire* Broadly, its function was to give a syst'ematic check 
on the researchers' impressions of what personnel in various sub- 
groupings actually do* More specifically, it represented an effort 
to determine which tasks in a given job classification are performed 
most frequently, whether there are differences across agencies or 
grade levels, which tasks are performed in some installations or 
specialties and not in others, "and whether there are differences 
in the kinds of tasks performed by the three ethnic subgroups 
studied* 
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Items for the Medical Technicians* checklist were selected from 
those developed in a study conducted by Morsh and Christal at the Air 
Force Perjsonnel ^Research Laboratory (1966). The task list >|ojp Carto- 
graphic Technicians was compiled from job specifications, personal 
observations, and interviews with technicians and supervisors, and 
wa6 administered by mail as part* of the preliminary Personal Jli^9ry 
Questic-".naire, previously described; The Inventory Management- 
Specialist task list was developed from interviews with incumbents 
and their supervisors', ^from job descriptions obtained during visits 
to agencies, from Civil Service Position Clajpsif ication Standards, 
and from the Daily Activity Log, previously described. 

The task list for Medical Technicians contained 58 iSems, for 
Cartographic Technicians, 130 items, and for Inventory Management 
Specialists, 184 items, reflecting the relative complexity of the 
jobs. For each task listed, the Medical Technicians and Cartographic 
Technicians were Instructed to check "Often," ''Sometimes, '||^eldom," 
or "Never," to indicate how often they performed the given task. The 
Inventory Management Specialists recorded the frequency with which 
they performed a .task as follows: "5 - Significant part of my job 
every day," "4 - Substantial part of my job, at least several times 
a week," "3 - Part o'f my job, probably once a week or twice a month," 
"2 - Part of my job, but seldom have to do' this, perhaps every month 
or at 'some regular time of year," "1 ~ Not a regularly assigned 
responsibility, but I sometimes do this," "0 - Definitely not part 
of ray job; does not apply." 

41 
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F. Pretesting of Materials 

* 

For all three phases of the study, newly developed instruments 

were pretested to determine optimum order of presentation, timing, 

•placement of re3t breaks, clarity of instructions, and potential 

« * 
problem are^fe. 

The aptitude test battery and peer rating scales for Medical 
Technicians were administered to all available subjects at the 
Veterans Hospitals in Wilkes^-Barre, Pennsylvania, and East Orange, 
New Jersey, prior to the full-scale testing program. Uork Sample 
tasks weres pretested at the Lyons, New Jersey, VA hospital. 

One hundred' forty-seven Job Knowledge Test items were pretested 

on a total of 26 technicians at two VA hospitals. From timing infor- 

^ J 

, mation gained irr these two test administrations, a, seventy-five item 
"test was assembled, with only the first sixty-three items to be 
scored. The remaining twelve items were included is a "filler" to 
occupy those technicians who completed the test inost quickly. 

The 165 items written for the Cartographic Technicians Job Knowl- 

■ ' / 

edg*e Test were divided into two test forms ta Shorten time needed for 
administration. These were pretested at the ^Providence, Rhode Island, 
field office of U; S. Army TOPOCOM. - Each 'form was given to 40 Carto- 
graphic Technicians, about^ half of whom were currently working ifi the 

>^ 

cartographic division and half in. the photogrammetric-, <iivision. Based 

* • *^ 

on how well the items discriminated between the high- end low-scoring 
groups and the total number of technician^ getting an item correct, 
approximatel}^ half of all the items contained in both forms vcre then 
chosen for the 75-item final form. Items dealing with the triangulation 
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specialLy were dropped, since the number of incumbents currently work- 
ing in chat specialty was small. 

After the three Cartographic Technician Work Sample tasks had 
been selected, each was tried out on a small sample of technicians at 
U. Army TOPOCOM in Bethesda, Maryland, and Providence, Rhode Island, 
to verify its*^ appropriateness and to determine the range of time 
required for completion. Each of the three tasks selected required 
about one hour to finish. 

The Inventory Management Specialist Work Sample was pretested 
in preliminary form at the Defense Personnel Support Center in 
Philadelphia. It was administered to 49 Inventory Management Sp^cial- 
'ists from the Medical, Subsistence, and Clothing' Directorates, and 
included Black and Caucasian male and female subjects at the GS-9 and 
-li salary grade levels. The original time allotted for pretesting 
was three hours. As a result, of the administration, instructions were 
^' changed and augmented, and information iti several problems was made 
•more specific. The actual working, time on the problems was increased 
from: two hours to two and- onp- I;-If hours, m^aking the total testing 
' time for the work sample exercise three aod one-half hours. 

The Criterion Rating Scales were admin: stered to supervisors of 
'the Inventory Management Specialists in the pretesting sample. , After 
each supervisor completed hfs ratings, another task was presented. 
For each Inventory Management Specialist he rated, he was asked to 
select the one trait he felt was most important to that person's job. 
This was not necessarily to be each manager's strongest trait, but 
rather the one most netessary for his success on his particular job. 
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As a result of the pretesting, this task was dropped as being difficult' 
for the supervisors to do and. not very meaningful in terras of the actual 
ratings. The Importance Rating Scale, previously described, was given 
instead in the main administration* 
" G«* Description of Samples Obtained and Data Gathered 

From the pretesting of materials, useful observations were made 
for testing practices which were incorporated in the main administration 
The following general procedures were followed for data collection: 

Testing was done on site. Al] tests, rating scales, and question- 
naires were administered by specially trained ETS staff members. This 
was felt to be necessary in order to maintain the confidential nature 
of the data, to see that standard directions were followed, and to 
ensure that maximum rapport was established. 
^ • Before the main test administration for each of the three occupa- 
tions, a letter was sent from the central agency office in Washington 
f 

to the director of each installation selected, explaining briefly the 
nature and purpose of the study and the respective roles of ETS and 
the U. S. Civil Service Commission, and expressing general support 
for the project. A suggested letter for notifying participating 
incumbents was sent to each installation, , to go out over the director's 
signature in advance of the actual testing session. In addition to 
outlining the purpose of the study and the naj:ure of their participa-- 
tion, it emphasized that the research was not identified in any way 
with formal agency personnel records or practices, and that testing 
materials and individual .results would be handled directly by ETS 
representatives on a confidential basis, with a guarantee that no 

J 



information about any individual would be made known to any government 
official or employee and that only overall results would be reported. 
Full sanction was to be given-'to.-.the research effort, including time 
taken off the job to participate. Although the addressees' personal 
cooperation and support were solicited, freedom not to comply was to 
be given. 

Every effort was made by ETS researchers to preserve the confi- 
dentiality of the data and to communicate this effort to the incumbents 
being, tested. Tight security precautions were taken with both used 
and unused tests and rating scales. At the testing sites, boxes were 
stored in locked rooms, and considerable precaution was taken not to 
leave the materials unattended during an administration. When tests 
or *ratihgs were completed, subjects personally placed all materials in 
individual envelopes arid sealed them* ETS reseai^chers were solely 
responsible for removal of testing materials from the premises. 

Fi^r the Medical Technician study, a group of 33 of the 170 
Veterans Administration hospitals was chosen. Most were selected 
because they employed large numbers of Blacks in their laboratories, 
a few because they were entirely or predominantly Caucasian, and 
about five because ^they were small. An effort was made to include 
hospitals' in various geographic locations. It was hoped to' test all 
Medical Technicians (GS-645) at these hospitals. Tallies indicated 

r 

that there were 774 technicians, including 261 Blacks. Of these, data 
were obtained on 465, of whom 168 were Black. The greatest attrition 
was due to the fact that technicians listed as working in particular 
hospitals actually worked in affiliated outpatient clinics, research 



groups, and the like. Second largest cause of attrition was that many 
of the technicians iworked only part time or were on evening hours, and 
CQuld not reasonably be included in the test sessions. The remaining 
attrition was due to such expected reasons as vacations, illnesses, 
being needed in the laboratory, and, in only a few cases, ^individual 
refusal to participate in the study* * 

Total time allotted for- the administration of the Aptitude and 
Job Knowledge tests, Personal History Questionnaire, and Peer Rating 
Scales, was approximately six ^hours per technician, with- another 2-3 
hours needed for those participating in the Work Sample tasks. Most 
researchers took a full week to gather data from one hospital, since 

the technicians were generally available for testing during the after- 

A, 

noons onlyr because of morning time pressures on their jobs. Often 

it was possible to test only about half the personnel from a laboratory 

at one time, since no more could be spared from their duties. 

For the Cartographic Technician study, a sample of approximately 
440 was tested from.l,OpO in the GS-1371 classification at four U. S. 

Army TOPOCOM locations. Technicians were selected from those who had 

> 

filled out the Preliminary Personal History Questionnaire, plus a few 

^. 

newer employees • The to^tal sample consisted of 101 Black, 99 Mexican- 
s- 
American, and 241 Caucasian technicians, and reflected the proportions 

of technicians workiiig in each, of the three major specialties, carto- 

"graphy, photdgrammetry, and triahgulation*. 

All available technicians at San Antonio (where nearly all of 

the Mexican-American technicians were employed) were tested. At the 

other three locations, all Black technicians available during the 



testing period were included in the sampl^e. Caucasian technicians 

were selected who were similar in GS-level to that of the Black 

. • t 

/ 

technicians/ and ^at a ratio of about two Caucasians for every 
Black. The resulting sample at the four locations included 167 

-technicians of 564 employed at .Washington,- 77 of 146 at Kansas ^ 
City, 75 of 143 at Louisville, and 122 of 130 at San Antonio. 

-Tlme/required to administer- the aptitude battery. Technical 

,Knowledge Questionnaire, and the Personal History-Questionnaire, 
was about five hours. Each Work Sample task required one hour, 
making the total testing time about eight hours. - The* Work- Sample 
was administered on a half -day following the other tests. 

. A sample of Cartographic Technicians at the Coast & Geodetic 
Survey was used in a supplemental study. Subjects were given all 
of the aptitude, tests and they^ersonal History , Questionnaire, and 
supervisors* ratings were ^obtained, but the Technical Knowledge 
Questionnaire and the Work Sample were not administered. Most of 
the work in this agency involves the production of aeronautical 

' and navigational charts, and supervisors, after careful review, 
advised that these measures ^.^ere not appropriate for their 
installation. 

For the Coast & Geodetic Survey sample in Silver Spring, 
Maryland, all available Cartographic Technicians were tested, a 
total of 98. This number included nine Caucasian deaf-mutes. 
This group possibly had difficulty in understanding the directions 
given for the tests, although the directions were interpreted in 
sign language for them. A preliicinary analysis showed that their 



mean test scores .were substantially lower than the means for the others 
tested, so this group was excluded from further analj^es. 

For the Inventory Management Specialists, plans were made to test 
approximately 200 Blacks, 100 Mexican-Americans, and .300 Caucasians at 
five locations. The sample obtained was smaller than anticipated, as 
shown in Table IIt5 on page 43. When the testing began, much greater 
resistance was encountered than in the previous phases, although every 
effort was made in advance contacts to provide information about the 

i 

.project and to encourage participation. 

'I 

A decision was made to test primarily at grade levels 9 and 11, 
the j^ourneyman levels in inventory management/after progress through 
the GS-5 and -7 training periods. (Entry into the 2010 classification 
is at grade 5, with progress to grade 7 and then to grade 9 within a. 
prescribed period, subject to satisfactory performance.) A few inven- 
tory managers in GS-7 wetti included in order to increase tHe ethnic 
samples. ^ 

Time required to administer the ^ titude battery .and the Personal 
History Questionnaire was about six and one-quarter hours. The Work 
Sample exercise, which was administered on a half-day following the 
other tests, required three and one-half hours, making the total test- 
ing time about ten hours. 
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Table II-l shows the number, and corresponding percentages of 
the total samples, of Medical Technicians tested at each grade level 
by jethnic group, and total number of subjects tested. 

Table II-l 

Distribution .of Medical Technicians Tested, 
by Grade^^Level and Ethnic Group 

Grade. Level Black % Caucasian % Total 



GS-4 and below 


'20 ' 


11.9 


29 


9.8 


49 


* 5- 


46 


27.4 


71 


23.9 


117 


6 


61 


36.3 


123 


41.4 


184 


7 


35. 


20.8 


-.58 


19.5 


93 


8 and above 


6 ' 


3.6 


16 


5.4 


22 


Total 


168 




297 




465. 
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Table II-2 describes the sample of Cartographic Technicians'^ tested, 
by location, ethnic group, and total • 

* Table II-2 
Distribution of* the Sample of Cartographic Technicians 
Tested, by Ethnic Group and Total 



S. Army 
TOPOCOM 



Black 



Mexican- 
American 



Caucasian 



Total 



* Washington 
Kansas City 
Louisville 
San Antonio 

Total 



53 
21 
25 
2 

101 



0 
0 
0 

9^ 
99 



114 
56 
50 

✓ 

- 21 
241 



167 
77 
75 

122^ 

441 



Coast & 
Geodetic 
Survey 

Silver Spring 



38 



60* 



* Includes 9 deaf-mutes, excluded from the comparative analyses 
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*- Table II-3 shows the distribution of Cartographic Technicians tested 
by grade level, percentages by ethnic group, and total number of subjects 
tested, at all TOPOCOM installations included in the study. 

Table II-3 

Distribution of Cartographic Technicians Tested 





at 


U. S. Arny TOPOCOM. Installations, 










bv dradi"- 


LpvpI and 


Ethnic Group 






ide Level 


Black 


% 


Mexican- 
American 


% Caucasian 


% 


• 

Total 


GS-5 ■ 


0 




0 ■ 


12 


5.0 


12 


6 


0 




0 


1 


.4 


. 1 


7 


32 


32.3 


17 


17.3 , 56 


23.4 


105 


8 


10 


10.1 


0 


19 


8.0 - 


29 


9 


51 


51.5 


81 


82.6 131 


54.8 


263 


10 


0 




0 


1 


■ .4 


l" 


li 


5 


5.0 


0 


19 


8.0 


24 


12 


1 


1.0 


0 


0 

> 




1' 


Total 


99 




98 


239 




436 



The numbers in Tables II-2 and -3 differ slightly because of missing data 
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Table II-A^^sh^ws the distribution of Cartographic Technicians tested 
by grade level, percentages by ^ethnic group, and total number of subje^s 
tested, at- the Coast & Geodetic Survey. ^ 

Table II-4 

Distribution of Cartographic Tec?hnicians Tested 
at the Coast & Geodetic Survey, 
by Grade Level and Ethnic Group 



Grade Level 


Black 


% 


Caucasian 




Total 


GS-2 


0 * 




2 


3.3 


2 


3 


1 


2.6 


6 


10.0 


7 


4 


0 




5 


8.3 


5 


5 


1 


2.6 


7 


11.7 


8 


6 


10 


26.3 


9 ■ ■ 


15.0 


19 


7 


9 


23.7 


5 


8.3 


14 


9 


13 


34.2 


18 


30.0 


31 


11 


4 


10.5 


8 


13.3 


12 


Total 


38 




60 




98 



52 
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Table II-5 describes the sample of Inventory Management Specialists 

4 

tested, by location, ethnic group, and total, 

\^ Table II-5 

Description of the Sample of Inventory Management Specialists ^ 
Tested, by Ethnic Group and Total 

Mexican- 
Installation Black American Caucasian Total 



Army Tank-Automotive Command, ^2 0 49 91 

Detroit 

Defense Electronic Supply Center, 0 ^44 80 

- Dayton " . ' 

Defense Industrial Supply Centei:, q 3^ 45 

Philadelphia - 

Navy Aviation Supply Office, g q 15 % 21 

Philadelphia # 

San Antonio Air Materiel Area, 75 ^gg 
San Antonio (Air Force) 



Total 



116 75 214 405 



Table II-6 shows the distribution of Inventory Management Specialists 
tested by grade level, percentages by ethnic group, and total number of 
subjects tested • ^ 



Table II-6 

Distribution of Inventory Management Specialists Tested, 
by Grade Level and Ethnic Group 



Grade Level 



I 
i 

Black 



•Mexican- 
American 



% Caucasian % 



Total 



GS-7 7 


6.1 


14 


18.9 


8 


. 3.9 


29 


9 76 


66.7 


50 


67.6 


136 


66.3 


262 


11 24 


21.0 


6 


8.1 


41 


20.0 


71 


Other (or missing data) 7 


6.1 


-4 


5.4 


20 


9.8 


31 


Total 114 




74 




205 




393 


The grade level data in Table II-6 


are 


taken 


from the 


Personal 





History Questionnaire, not completed by a small number of subjects, 
which explains the difference xn the Ns in the two tables. 
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H. Reporting Individual Test Results to Subjects 

Individual test scores were not made available to participants in 
the Medical Technicians study. Inventory Management Specialists were 
given the option of receiving their scores on the aptitude tests and 
Cartographic Technicians on the aptitude tests and Job Knowledge Test. 

o 

Supervisors' Ratings and Work Sample Test scores were* not reported to 
'any ^individuals. . ^ 

A letter outlining factors that might influence test scores and 
giving a brief description of each test and^jihart---xt'l7as intended to 
measure accompanied the individual test results. Score reports were 
sent to -home addresses given by those who elected to receive a report 
as a guarantee .of confidentiality. , 

Scores were reported as percentiles, based on the total sample 
tested at all installations. An interpretation of percentile rank- 
ings was given, using the following guidelines: 

Percentile Interpretation 

15 and below Low 

' 16-30 Below Average 

31 - 69 Average 

70 - 84 ^ Above Average 

85 and above High 
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Chapter III 

Comparison of Ethnic Groups .on Aptitude Measures 
The aptitude and ability measures selected as the predictor bat- 
teries for the three occupations studied are described in detail in . 
the instrumentation reports previously cited (Pike, 1969; Parry, 1971; 
Crooks and Mahoney, 1971). In this chapter, the measures will be de-< 
scribed briefly in relation to factors observed as directly related Co 
job performance in the three occupations and to other factors of gen^ ^ ^ 



^eral research interest, as background for discussion of^he results. 
The means and standard, deviations of ^he scores obtained on the mea- 
sures will be compared by ethnic group for each- of tha\three studies. 

Medidal Technicians - ' 

^ - - -* 

Nine tests were selected to comprise the predi^df^ battery for 
study of tfie occupation of Medical Technician:* ^ 
% . R?.ctor ' Name of Test ' ^- • 

Number Facility Subtraction & Mul triplication ^ 

Facility in performing-^basic arithmetic compntatioKs 
Perceptual Speed Number Comparison 



Accurate performance of clerical tasks under 
pressure 

Flexibility of Closure Hidden Figures 



Ximei 
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Ability to identify known configurations when they , 
£.re presented with perceptual distractions 

Speed of Closure Gestalt Completion 

<- 

Ability to organize and recognize an apparently * 
disparate field as a single percept 

Both Flexibility of Closure and Speed of Closure are sug- ' ' ^ 
gested a^y related to examination ^of laboratory '-.peciraens. /^''^ 

-47- 
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'\ ^ - , 

Factor Name of Test • , 

Visualization . Paper Folding 

Related to -mechanical ability and comprehension in 
setting up, calibrating, adjusting, and maintaining 
laboratory equipment 

Fine Finger Dexterity Pin-Dexterity 

(USCSC Test No. 26) 

Skill at hand' manipulation of instruments and 
jjiaterials under time pressure ' * 

Verbal Comprehension " Extended Range Vocabulary 

Ability related to successful performance in a wide 
range of jobs (tests of this factor have been 
criticized as biased against minorities) 

J ' • 

.Assojciative (Rote) Memory Picture-Number 

A general ability to be explored as related to job 
perfprmance of Medical Technicians 

General Reasoning . Necessary Arithmetic Operations 

A general ability factor to be explored as related 

to job performance ' • ,> 

Ethnic Group Comparisons (Black - Caucasian) 
> *Table III-l shows the means and standard deviations of these mea- 
sures for the Medical Technicians by ethnic group (Black and * Caucasian ) . 
On all meafeuiTes, the Black group scored lower on the average. On all 
but one of tiie nine measures, mean score differences were significant 
at the .01 levelv On the other test, Pin-«Dexterity (measuring fine 
finger dexterity), the mean difference was significant at the .05 level. 

,Jt will be seen that all but two differences were apprc:nmat€jLy One- 
half standard deviation in size. The, mean difference on the Picture- 

'Number T-est (a measure of rote, short-term memory) was less than one half 
standa^r4 deviation. The mean difference on the Necessary Arithmetic 
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Operations Test (a test of general reasoning) was approximately one - 
standard deviation. 

The interccrrelations of the tests in the predictor battery are 
shown in Appendix Table III-A; by ethnic group, if of interest. Some 
difference in size of correlations can be observed between the two 
ethnic groups^ which should be kept in mind in considering the results 
of further .analyses described in later chapters. 
Cartographic Technicians 

.Thirteen tests were selected to comprise the predictor battery for 

the study of the occupation of Cartographic Technician (map maker): 

Factor Name of Test 

Hand-Arm Movement Flanagan Coordination 

Coordination 

Facility in coordinating hand-arra movements in 
drawing and drafting 

Flexibility of Closure Hidden Figures 

Ability to extract detail from aerial photographs 
of terrain with indistinct or distracting background 

Verbal Comprehension Vocabulary (speeded) 

Extended Range Vocabulary (unspeeded) 

Ability to understand written instructions and 
specifications 

Associative (Rote) Memory Object-Number 

Ability to form and remember new associations; 
raeraory for specifications 

Spatial Orientation Card Rotations 

Ability to perceive spatial patterns and maintain 
orientation of objects in space / 
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Factor Name of Test 



Arithmetic (USCSC Test No, 24) 

Ability to make numerical computations (i.e,, 
measuring distance to scale) 

Spatial Scanning Map Planning^ * ' (speeded) 

Maze Tracing Speed (unspeeded) 

Ability to explore visually a wide or complicated 
spatial field 

Visualization ' Surface Development 

Ability to ^manipulate the image of spatial patterns 
into other visual arrangements 

Following Oral Directions ' Following Oral Directions 

(USCSC Test No. ^35) 

Ability' to "J^arry out simple and complex instructions 
accurately when given orally and under time pressure . 

• Perceptual Speed ^ Identical Pictures 

Ability to carry out close, accurate visual tasks 
under time pressure 

^ General Reasoning Necessary Arithmetic Operations 

A general reasoning ability found to be important 
to performance in many jobs 

^ * The results for the two groups of Cartographic Technicians (TOPOCOM 

and Coastv & Geodetic Survey) are shown and discussed separately. The 

work performed by the technicians in these agencies differs considerably, 

and iz was felt that the results should be analyzed independently. Table 

III-2 shows the means and standard deviations of the scores on the pre- 

die tor battery for the TOPOCOM Cartographic Technicians by ethnic group 

(Black, Mexican-American, and Caucasian). Table III-3 shows the results 

for the Coast & Geodetic technicians (Black and Caucasian only). 
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Ethnic Group Comparisons (Biack - Caucasian) 
In comparing the mean scores of Blacks and Caucasi-^ns for the 
.NCPbCOM sample, it will be seen that mean score differences were sig- 
nificant at the .01 level on seven of the thirteen tests, and at the 
.05 ievel on four of the remaining tests. The mean scores of the 
BlacksXwere in all cases lower. On most of these tests, the differ- 
ences ranged from one-fourth to one-half standard deviation in size. 
On two tests. Vocabulary and Extended Range Vocabulary, the mean 
differences were small and nonsignificant. 

In comparing the mean scores of Blacks and Caucasians for the 

j 

Coast & Geodetic Survey sample, it will be seen that mean score dif- 
ferences were significant at the .01 level on eleven of the thirteen 
tests and at the .05 level on the other two. The Black sample scored 
lower on all tests. The mean differences ranged from one-half standard 
deviation to one standard deviation in size. The largest differences 
w^re on Vocabulary, CS Arithmetic, Following Oral Directions, and ^ 
l^ecessary Arithmetic Operations, all approximating one standard devi- 
ation in size. ^ ^ 

Some differences in the two samples of Black Cartographic Techni-- 
cians are apparent. The Coast i Geodetic Survey sample scored lower on 
the average on every test than the TOPOCOM sample, while the Caucasian 
sample on the average scored approximately at the same level. Differ- 
ences in background v^^riables such as level of e.ducation and experience 
for the two Black samplies may account for some of the variation. 

In seleg^ting. aptitude tests for the study of Cartographic Techni- 
cians, it was suggested that score differences between ethnic groups 

i 
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usually found in testing situations might be attributable at least in 

part to "testwiseness" or practice in taking tests in favor of the 
^ majority group, particularly aOxVantageous to this group under speeded 
conditions. To test the validity of this suggestion, it was decided 
to select two pairs of tests measuring common factors, one of eacn 
pair to be administered under regular, speeded conditions and the 
other to be made essentially unspeeded by extending the usual time 
limit. One such pair was 'chosen under the Spatial Scanning factor, 
with the Map Planning Test given as a speeded test and the Maze Trac- 
ing Speed. Test given as essentially unspeeded ./'The relative mean 
score differences on these two tests for the TOPOCOM Black and 
Caucasian samples do not show any notable change under speeded and 
unspeeded conditions. The mean score differences and the standard 
^ deviations are almost identical for the two groups on the two tests. 
The other pair of such tests Wasures a Verbal Comprehension 
factor. The Vocabulary Test was given under speeded conditions and 
the Extended Range Vocabulary Test was given as essentially unspeeded 
by extending the time limit. As seen from Table III-3, the mean 
scores and standard deviations were not significantly different under 
speeded and unspeeded conditions. 

For the Coast & Geodetic Survey sample, the mean scores of the 
Black sample on the tests given under speeded and unspeeded conditions 
for the Spatial Scanning factor do not appear to differ in relation- 
ship to the mean scores of the Caucasian sample (Table III-4). Each 
. set of scores differs by more than one-half standard deviation. On 
the Vocabulary (speeded) and Extended Range Vocabulary (unspeeded) 
/ 
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Tests, the mean score under the unspeeded condition appears to favor the 
Black sample. The mean score of the Black sample under the speeded con- 
dition was slightly more than one standard deviation below the Caucasian 
mean score, while on the Extended Range Vocabulary Test, essentially un- 
speeded, the mean difference was closer to one-half standard deviation. 
Ethnic Group Comparisons (Mexican-American - Caucasian) 
In' comparing the mean scores and standard deviations of Mexican- 
American and Caucasian technicians employed by TOPOCOM, it will be seen 
that the mean score for the Mexican-American sample was higher than for 
the Caucasian sample on one test of tlie thirteen in the battery, the 
Object-Nilmbe.r Test (measuring the ability to form and remember^ new 
associations), although this difference was small and not significant. 
On one other test. Identical Pictures (measuring the ability to carry 
out close, accurate visual tasks), the mean difference, favoring the 
Caucasian sample, was also small and not significant. The mean score 
differences on the remaining eleven tests were significant at the .01 ^ 
level. In all cases the Caucasian mean was higher. On eight of the 
tests, the diffe rence s in mean score_s ranged in magnitude from about _ 
one-third to one-half standard deviation. On three of the tests. 
Following Oral Directions, Extended Range Vocabulary, and Necessary 
Arithmetic Operations,, the size of the mean score differences was 
approximately one standard deviation. 

On these three tests, the Mexican-Americans may .have scored lower 
because they are less fluent in the English language. One hundred 
percent of the Mexican-Americans reported on the Personal History 
Questionnaire that a foreign language was spoken at home (50% said 
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"almost all of the time") . The Mexican-Americans also scored lower on 
the average |han the Black Cartographic Technicians on both Vocabulary 
tests, as well as on. the Following Oral Directions Test (where^'quick 

listening comprehension is needed), althqjagh .they scored as well or 

If 

better on most of the other tests not involying language to as great 
a degree. 

On the paired speeded-unspeeded tests, the speeded condition 
appeared to make no appreciable difference in the scores on the Spatial 
Scanning factor!- On the Extended Range Vocabulary Test, given under 
unspeeded conditions, the mean score of the Mexican-American sample was 
somewhat lower than on the speeded Vocabulary Test. This probably re- 
flects the wider range of the difficulty of the Extended Range Vocabulary 
Test, a handicap to those 'less fluent in English which would not be over- 
come b}' extending the time limit. 

Intercorrelations of the predictor battery for the TOPOCOM and 
Coast & Geodetic Survey samples are included in the appendix as Appendix 
Tables III-B and III-C. The correlations of the speeded and unspeeded 
tests for^the Spatial Scanning factor range from .43 to .61, and on the 
Verbal Comprehension factor from. .77 to .86 for the separate samples and 
ethnic groups. 

Inventory Management Specialists 

Twelve aptitude and ability measures were selected for the predictor 
battery for the occupation of Inventory Management Specialist: 
F actor Name of Test 

Perceptual Speed Number Comparison 

Speed and accuracy in comparing sets of numbers 
and identifying differences 

G3 
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Factor ' - Name of Test 

Flexibility of Closure Hidden Figures 

Ability to- identify known configurations when they 
are presented with perceptual distractions, sug- 
gested as a measure of analytical functioning ^ 

Verbal Comprehension Vocabulary. (speeded) 

Extended Range Vocabulary (unspeeded) 

Ability to understand the English language (Inventory 
Managers issue and receive both oral and written 
instructions of varying compl-exity) 

Associative (Rote) Memory, Object -Number^ 

Ability to remember bits of unrelated material 
(memory for large amounts of information is 
important to Inventory Managers) 

Induction . Letter -Sets 

Ability to form and try out hypotheses to find 
general concepts to fit sets of data 

Syllogistic Reasoning Inference (speeded) 

Nonsense Syllogisms (unspeeded) 

Ability to reason from stated premises to their 
necessary conclusions 

Number Facility Subtraction &- Multiplication 

Ability to do arithmetical operations with speed 
and accuracy 

General Reasoning ' Necessary Arithmetic Operations 

General reasoning ability, related to performance 
on a wide range of jobs 

Following Oral Directions Following Oral Directions 

(USCSC Test No. 135) 

Ability to carry out simple and complex instructions 
accurately when given orally and under time pressure 
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Factor Name of Test 

General Reasoning Portions of the Federal Service 

Entrance Examination (USCSC 
Test No* 170) ' 

Verbal ability, abstract reasoning ability, and » 
numerical reasoning 

Table III-4 shows the means and standard deviations of the aptitude 
test, scores for the Inventory Management Specialists by "ethnic group 
(Black, Mexican-American, and Caucasian). 

Ethnic Group Comparisons (Black - Caucasian) 

The mean score differences on nine of the 12 tests in the battery 
were significant at the .01 level, approximately one-half standard devi- 
ation in size* On one test. Necessary Arithmetic Operations (a test of 
general reasoning), the difference approached one standard deviation. 

For the other three tests, Niimber Comparison, Object-Number, artci Non- 

/ 

sense Syllogisms, the mean differences were small and nonsignificant. 

/ 

However, the mean scores of the Black sample were lower on' all tests. 

For- the speeded pair of tests under the Syllogistic Reasoning 
factor (Inference and Nonsense Syllogisms), the speeded condition ap- 
peared to make a difference, with the Black sample^ scoring significantly 
lower on the average than the Caucasian sample on the Inference Test, 
while on the Nonsense Syllogisms Test, given essentially unspeeded, the 
mean difference was small and nonsignificant. 

Ethnic Group Comparisons (Mexican-American - Caucasian) 
The mean scores of Mexican-American Inventory Management Special- 
ists w.ere higher on two of the 12 aptitude measures: the Number 
Comparison Test (measuring perceptual speed), where the mean difference 
was not significant, and the Object-Number Test (measuring associative 
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(rote) memory), where the mean difference was significant at the .01 
level. On the other ten tests, the Caucasians had higher mean scores, 
although the mean difference on the Hidden Fi:5ures Teat was not signif- 
icant. For three tests. Letter Sets, Subtraction & Multiplication, and 
Necessary Arithmetic Operations, the mean differences were significant 
at the .05 level, and for six tests, the mean differences were signify 
icant at the .01 level. The size -of the significant differences ranged 
from one-fourth to one-half standard deviation. 

On the speeded-unspeeded pairs of tests under the Syllogistic 
Reasoning factor and the Verbal Comprehension factor, the mean differ- 
ences were*^about the same under both conditions. It may be noted that 
the Mexican-Americans were again shown to be at a disadvantage on tests 
involving language, as seen in the study of Cartographic Technicians. . 

Intercorrelations of the predictor battery\for Inventory Manage- 
ment Specialists are included in the appendi^as Appendix Table III-D. 
The intercorrelations of the speeded-unspeeded pairs of tests range 
from .30 to .56 for the Syllogistic Reasoniag factor and from .73 to 
•88 on the Verbal Comprehension factor. 

Cross-Study Comparisons 

Three tests, were included in the batteries for all three occupa- 
tions: Necessary Arithmetic Operations, Hidden Figures, and Vocabulary. 
Table III-5 shows the means and standard deviations for these tests by 
occupation, by ethnic group. Tables III-6a and III-6b show the means 
and standard deviations for the five tests included in the predictor 
batteries for two occupations by ethnic group. 

The mean scores on these tests vary somewhat across the three 
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occupations, perhaps reflecting individual differences in aptitudes 
or abilities needed from occupation to occupation, background factors 
such as amount of education or special training required, GS level, 
and restriction of range as a result of preselection or self-selection. 
Later analyses will show how these differences relate zo measures af 
performance. 

Reliabilities 

Tables III-7, III-S, and III-9 show the alternate form reliabil- 
ities of the aptitude tests and as corrected for attenuation by the 
Spearman-Brown formula for the occupational samples by ethnic group. 
In general, the reliabilities of the tests do not differ to any marked 
degree by ethnic group and are of acceptable size. 

Summary 

Black - Caucasian Comparisons 

For all three occupations, Caucasians consistently scored higher 
\ on the average than Blacks on the aptitude mPAsures. For the Medical 
Tecbtii^'rian samples, all mean score differences were statistically 
significant. For the Cartographic Technician samples, the mean score 
differences of 11 of the 13 tests were statistically significant. For 
the Inventory Management Specialist samples, the mean score differences 
on nine of the 12 tests were statistically significant. Differences 
ranged from on^-fourth to one-half standard deviation on most tests, 
with the exception of the Necessary Arithmetic Operations Test (a test 
of general reasoning). On this test, for all three occupations, the 
differences approached one standard deviation. 

The differences in the test scores of the three samples of Blacks 
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and Caucasians are smaller t found in other populations, and it is 
suggested that these samples represent selected groups of Blacks and/ 
or Caucasians. The differences in "the test results vary among the 
three samples, also suggesting that the special aptitudes or skills 
needed for the three occupations may result in greater selectivity 
among the Blacks for some jobs than for others. There was little, if 
any, explicit pre-employment selection on tests for the Medical Techni- 
cians and Cartographic Technicians. Thus, the fact that these groups 
appear to be somewhat selected is incidental to the other selection 
procedures or to retention factors on the job, or both. ^ 
Mexican-American - Caucasian Comparisons 

For the two occupations in which Mexican-Americans are represented 
Caucasians scored higher on the average than Mexican-Americans on most 
of the aptitude tests. However, the Mexican-Americans scored higher on 
the average than the Caucasians on tests of rote memory and perceptual 
speed. The other differences between Mexican-Americans and Caucasians, 
where significant, were of about the same magnitude as for Blacks and 
Caucasians, with the exception of tests involving vocabulary, where dif 
ferences approached one standard deviation. Blacks also scored higher 
than Mexican-Americans on these tests. 

Note: Differences between mean scores of ethnic groups on aptitude 
tests and on selected criterion measures are shown graphically in 
Chapter IV in Figures IV-1, IV-2, IV-3, and IV-4, in terms of standard 
deviation units from **he Caucasian mean. 
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Table III-l 
Means and Standard Deviations of 
Aptitude Test Scores by Ethnic Group 
Medical Technicians 



Black Caucasian 
N=168 N=297 . 



Tes.fc 

« 


Mean 


S.D. 


Mean 


S.D. 


Subtraction and Multiplication 


46.6** 


17.1 


57.0 


19.6 


Vocabulary 


19.2** 


7.7 


• 25.7 


9.1 


Hidden figures 


5.7** 


4.0 


7.8 


4.9 


Necessary Arithmetic Operations 


10.4** 


3.7 


14.0 


4.8 


Pin-Dexterity 


22.5*' 


14.7 


29.1 


15.8 


Number Comparison 


36.5** 


9.7 


42.4 


9.8 


Gestalt Completion 


10.3** 


5.2 


12.1 


5.6 


Picture-Number 


17.5** 


8.8 


20.8 


9.4 


Paper Folding 


6.8** 


i 3.3 


9.1 


3.8 



*Signif icantly different 
from Caucasian mean at .05 level 

**Significantly different 

from Caucasian mean at .01 level 
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Table III-2 
Means and' Standard Deviations of 
Aptitude Test Scores by Ethnic Group 

Cartographic Technicians (TOPOCOM) - — 
Black Mexican-American Caucasian 





N=101 


N=101 




N= 


=240 














V 


Test 


Mean 


S.D. 


Mean 


S:d. 


Mean 


S.D. 


Coordination 




1 H 
X . 0 


5.0** 


2.0 


6.0 


2.1 


Hidden Figures 




D • 0 


. 9.4** 


5.6 


11.9 


6.3 


Vocabulary 


19.0 


7.4 


15.4** 


6.2 


19.9 


7". 3 


Ob 3 e c t — Numb e r 


10.0** 


4.9 


12.4 


-6.3 


12.1 


6.7 


Card Rotations 


112. 7* 


39.0 


108.9** 


41.5 


124.0 


41.5 


CS Arithmetic 


35.2** 


"9.1 


37.2** 


8.8 


40.9 


( ft.O 


Map Planning 


16.9* 


6.9 


16.5** 


7.0 


18 ..9 


6.8 


Surface Development 


24.0** 


■ 14.7 


22.8** 


14.0' , 


31 .'3" 


15.0 


Maze Tracing Speed 


30.1** 


8.1 


30.8** 


8.4 . 


34.7 


■ 8.9 


Following Oral 
Directions 


15.6** 


8.2 


11.0** 


7.4 


'20.2 


8.2 


Identical Pictures 


61.2** 


13.3 


62.5 


12.7 


65.3 


13.0 


Extended Range 
Vocabulary 


18.6 


9.6 


12.8** 


7.6 , 


19.9 


9.2 


Necessary Arithmetic 
Operations 


10.3** 


5.0 


10.6** 


5.0 


• 14 .'6 

\ 


5.4 



*Signif icantly different 
from Caucasian mean at .05 level 



**Signif icantly different 

from Caucasian mean at .01 level 
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Table III-3 * ^ 
-^■v Means and Standard Deviationa of 

Aptitude Test Scores by Ethnic. Group * 
Cartographic Technicians (Coast &'-<5eodetic Survey) ' 



Coordination" 



Test 



Hidden^Figures ' 

VSa«Pulary "^^ ' 

Object-Number 

Caxd Rotations 

CS^ Arithmetic 

Map Planning 

.Surface Development 

J^ze Tracing Speed ' ^ 

Following Oral Directions 

Identical Pictures 

Extended Range Vocabulary 

Necessary Arithmetic Operations 

*Signif icantly different 
frgp Caucasian mean at ,05 level 



Black 
N=38 



Caucasian 
N=51 ' 



**Signif icantly different ' 

6rom Caucasian mean at ,01 leve 



Mean 


-S.D. 


Mean 


S.D. 


5.2* 


1.9 


6.1 


2.2 


7.6* 


5.5 


10.0 


6.4 


1J.9** 


8.3 


.20.8 


6.4 


9.1** 


5.9 


13.9 


7.3 


p.o** 


47.4 


116.4 


41.3 


26.3** 


10.1 . 


38.2 


11.3 


f3.3** 


7.6 


17.4 


. 6.9 


17.7** 


11.7 


29.9 


^j^.3 


23.8** 


10.6 


30.9 


9.8- 


9.6** 


7.5 


18.8 


9.4 


•51.0**^ 


12.5 


'59.8 


12.7 


14.5** 


10.2 


20.0 


9.4 


7.6** 


4.8 


13.1 


6.3 


>• 









ERIC 
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Table III-4 
Means and Standard Deviatic^s of 
Aptitude Test Scores by Ethnic Group 
Inventory Management Specialists 



• Black Mexican-American Caucasian 





N=112 


N=72 






=194 


Test 


Mean 


,S.D. 


Mean 


S.D. 


Mean 


S.D. 


Number Comparison 


. 39.8 


'll.l 


43.6 


10.3' 


40.9 


11.7 


Hidden 'Figures 


5.2** 


5.3- 


7.0 


4.8 


7.5 


5.7 


, Vocabulary 


22.3** 


7.1 


21.2** 


5.9 


25.6 


6.4 


Object-Number 


10. 8 


6.4 


14.5** 


7.1 


11.2 


6.0 


Letter Sets 


12.8** 


6.0 


14.2* 


5.5 


15.8 


6.3 


Nonsense Syllogisms 


7.7 


7.3 






9 0 


8.2 


Subtraction and 
Multiplication 


63.9** 


19.2 ■ 


70.7* 


20.1 


76.6 


21.6 


/Extended Range 
-Vocabulary 


24.2** 


. 10.1 


21.0** 


8.8 


27.7 


10.4 


Necessary Arithmetic 
Operations 


11.1** 


5.6 


13.7* 


5.4 


15.5 


5:4 


- Following Oral 
Directions 


14.6** 


7.4 


14.9V* 


7.4 


18.1 


8.5 


Inference 


' 9.4** 


4.5 


9 . 2** 


4.1 


il.7 


4.9 


Federal Service 
Entrance Examination 


53.1** 


18.8 


57.6** 


14.5 


65.1 


18.7 



*Signif icantly different 
from Caucasian mean at .05 level 

**Signif icantly different 

from Caucasian mean at .01 level 
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Table III~7 - - 

Alternate Form and Spearman-Brown Corrected Reliabilati 
of Aptitude Tests by Ethnic Group 
Medical Technicians 
Black 



^AB 



SB 



Caucasian 



^AB 



SB 



Subtraction & 
Multiplication 

Vocabulary . 

Hidden Figures 

Necessary 

Arithmetic 

Operations 

Number 
Comparison 

Gestalt 
Completion 

.Pic tur e-Numb er 

Paper Folding 



.89 



,77 
.71 

.56< 



,65 

\ 

.5\l 



.94 

.87 
.83 

.72 



.90 .95 



.83 .91 

\ 

V74 .85 



.79 
.68 



.83 
.62 



.70 
.69 



.91 
.77 



,71 .83 



.82 .90 



.78 .88 



.82 
.82 
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Table III-8 

Alternate Form and Spearman-Brown Corrected Reliabilities 
of Aptitude Tests by Ethnic Group 
Cartographic Technicians 
TOPOCOM 



COAST & GEODETIC SURVEY 



Black 



Mexicans- 
American Caucasian 



Black 



Caucasian 





A5 


bB 


Ad 


bti 






"ab 


"SB 


"ab 


"sB 


Hidden Figures 


.48 


.65 


.42 


.59 


.62 


.77 ■ 


.69 


.82 


.72 


.84 


Vocabulary 


.77 


.87 


.74 


.85 


.77 


.87 


.78 


.88 


.83 


.91 


Obj ect-Nuraber 


.40 


.57 


.63 


.77 


.72 


.84 




.72 


.67 


,80 


Card Rotations 


.77 


.87 


.82 


.90 


.79 


.88 


.77 


.87 


.88 


.94 


Map Planning 


.70 


.82 


.64 


.78 


.63 


.77 


.74 


.85 

* 


.71 


.83 


Surface 
Development 


.83 


.91 


.78 

1 


.88 


.82 


.90 


.70 


.82 


.84 


.91 


Maze Tracing 
Speed •* 


.87 


.93 


.85 


.92 


.89 


.94 


.96 


.98 


.94 


.97 


Extended Range 
Vocabulary 


.79 


.88 


.76 


.86 


.85 


.92 


.79 


.88 


.90 


.95 


Necessary 

Arithmetic 

Operations 


.62 


.77 


.68 


.81 


.66 


.80 


.63 


.77 


.67 


.80 
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Table III-9 

Alternate Form and Spearman-Brown Corrected Reliabilities 
of Aptitude Tests by Ethnic Group 
Inventory Management Specialists 



6iack 



■AB 



i^exican— 
American 



^AB 



^SB 



Caucasian 



^AB 



^SB 



Number^ 
ComRkrison 

Hidden Figures 

Vocabulary 

Object-Number 

Letter Sets 

Nonsense 
Syllogisms 

Subtraction & 
Multiplication 

Extended Range 
Vocabulary 

Necessary 
Arithmetic ' 
Operations 

Inference 



.85 

.54 
.71 
.75 
.73 

.56^ 
.85 



.68 




.92 



.83 .91 



.81 



.59 .74 



.91 



,75 



,75 



,51 



..95 



.86 



.86 



.68 



.83 

.51 
.70 
.61 
.76 

.50 
.92 
.85 



.91 

.68 
.82 
.76 
.86 

.67 
.96 
.92 



.68 .81 



.68 .81 
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Chapter IV 

Comparison of Ethnic Groups on Criterion Measures 
A description of the selection and development of the criterion 
measures for the three occupations is given in detail in the instru- 
mentation reports previously cited (Pike, 1969; Parry, 1971; Crooks 
and Mahoney, 1971). In this chapter, the measures will be described 
•only briefly as background for discussion of the results. The ipeans 
and standard deviations of the scores and the intercorrelations; of 

the measures will be presented by ethnic group for each of the 'three 

I 

studies. / 

A •• / 

Medical Technicians j 

/ Two types of criterion measures were used in this study:* Super- 
f i 

visors' Ratings (eight scales describing aspects of performance, on the 

job and a rating of overall effectiveness) and a Job Knowledge Test. 

^ y' Peer ratings were obtained but not used in the analyses. 

Rating Scales ' 

\ 

,The rating scales are described below in behavioral terms: 

Flexibility \ 

Ability to adapt readily to a wide variety of tasks, to 
changes in procedures and equipment, and to emergency 
situations 

Orgajj^ization • I 

Skill in planning and organizing work, assigning priorities, 
and anticipating needs 



Interest 



Interest in w • and initiative in improving performance 
Learning Ability 

Willingness and ability to learn on the job; quick to 
understand 

-81- 
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Job Knowledge (Technical Knowledge) 

Knowledge and comprehension of the body of technical 
information and procedures surrounding the job ' 

Technique , 

^ Quality of technique in the laboratory in handling » 
equipment, materials, and tasks 

Low Need for Supervision ' 

Decree of responsibility assumed on the job without 
direct supervision 

Communication 

Ability to communicate with others 
Overall Rating 

Overall effectiveness as a Medical Technician 
Job Knowledge Test 

The Job Knowledge Test was developed to measure the knowledge of 
the Medical Technicians about their field (information and techniques) • 

Ethnic Group Comparisons j 

Table IV-1 shows the means and standard deviations of the, cr^Lte- 
rion measures for the Black and Caucasian samples in the Medical 
Technician study. 

Supervisors^ Ratings : For all but one of the rating scales, the 
mean ratings of Caucasians were slightly higher. The mean rating on 
the Flexibility scale was very slightly higher for the Black sample.^ 
The only significant difference in mean ratings (at the .05 level) was 
on the Low Need for Supervision scale ("low need" at the upper end of 
the scale). 

Job Knowledge Test : On the Job Knowledge Test, the mean score 
difference between the Black and Caucasian samples was significant at 
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the.Ol level. The mean of the Caucasian sample was approximately one- 
half standard deviation higher ♦ (It may be noted that the means of the 
supervisors' ratings on the Job Knowledge scale for the Black and Cauca- 
sian samples differed only slightly •) 

Figure IV-il shows -graphically the differences between Caucasian 
and Black mean scores for aptitude tests (see Chapter III) and .selected 
criterion measures for Medical Technicians (supervisors' ratings of } 
Learning Ability, Job Knowledge, and Overall effectiveness; and the Job 
Knowledge Test). The differences between the means are plotted in terms 
of standard deviation units, showing the Caucasian mean as point of 



departuife. 




Table IV-2 shows the intercorrelations of the criterion measures 
for the Medical Technicians by ethnic group. The intercorrelations of 
the ratings for both Black and Caucasian s^amples are quite high (ranging 
from .65 to .84 for the Black sample and from .55 to .77 for the Cauca- 
sian sample). The separate rating scales also correlate highly with the 
overall rating (?g), 'ranging from .70 to .81 for the Black sample and 
from .65 to .77 for the Caucasian sample. It was hoped that the "halo" 
effect usually found in such ratings would be less through the use of 
behavioral descriptions to anchor the scales. The Job Knowledge Test 
correlations with the rating scales range from .38 to .58 for the Black' 
sample and from .22 to 135 for the Caucasian sample. The highest corre- 
lations of ratings with the Job Knowledge Test for both samples are with 
the scales for Job Knowledge and Learning Ability, which might be con- 
sidered a kind of validation of the Job Knowledge Test. The reasons for 
the difference in the magnitude of the intercorrelations of the ratings 
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with the Job Knowledge Test for the Black and Caucasian samples are ob7 
scure, and later analyses may be enlightening. The differences suggest, 
however, that the two groupsT are being evaluated on somewhat different 
heses. 

Cartographic Technicians ^ 

Three types of criterion measures were used in the study of Carto- 
graphic Technicians: Supervisors* Ratings (seyen rating scales describing 
aspects of performance and a rating of overall effectiveness), a Job 
Knowledge .Test, and a Work Sample with three separate tasks. All three 
were included for the TOPOCOM sample, but only Supervisors*- Ratings were 
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obtained for the Coast & Geodetic Survey s&mple. The Job Knowledge Test 
and^ the Work Sample tasks were not considered appropriate for this group, 
for reasons discussed in Chapter II. 
Rating Scales 

The rating scales used to obtain supervisors' ratings for the Carto- 
graphic Technicians are describjedtelow in terms of behavior on the job: 
Accuracy 

Ability to produce accurate work, whether simple or complex 
I nterest 

Takes interest and pride in the work; improves through study 
and practice ^ 

Learning Ability 

Ability to understand new procedures and ideas with a minimum 
of instruction or explanation 

Job Knowledge (Technical Knowledge) 

Knowledge and comprehension of the body of technical information, 
and procedures in the cartography field 

82 ' 

J 




Dexterity 

Ability to work with tools and instruments with precision 
and* skill; eye-hand coordination 

Low Need for Supervision 

Ability to carry out and show initiative in work with 
minimum direction or supervision 

^ ' Perseverance 

Patience and stick-to-itiveness in working on painstaking 
tasks to completion 

> . Overall Rating 

Overall effectiveness as a Cartographic Technician 

Job Knowledge Test 

The Job Knowledge Test was developed to measure the knowledge of 
the Cartographic Technicians about their field (information and techniques 
relating specifically to the cartographic and photogrammetric aspects_^con- 
sidered to be the primary areas of work). 

^ Work Samples " ^ 

Three work sample tasks were developed for' th Cartographic Techni- 
cian study: a geometric restitution task, a logical contouring problem, 
and a "pull-up*' from an aerial photograph, as described fnore fully in 
Chapter II. The. scores for the separate tasks were standardized s^ they 
could be combined to provide- a Work Sample Composite score. 
Ethnic Group Comparisons (TOPOCOM) 

i 

Table IV-3 shows the means and standard deviations of the criterion 
measures for TOPOCOM installations,* which included Black, Mexican-American 
and Caucasian , samples. 

Black - Caucasian Comparisons 

Supervisors' Ratings: Caucasians received higher mean ratings on 
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all scales. On one scale, Interest (in the work), the mean difference 
was significant at the .01 level. On three other scales. Accuracy, 
Learning Ability, and Low Need for Supervision ("low need" at the upper 
end of the scale), the mean difference was significant at the .05 level. 
ihe differences were all less than one-half standard deviation in size. 
On three scales and on the Overall Rating, the mean difference was not 
significant. 

Job Knowledge Test : The mean score of the- Black sample on the Job 
Knowledge Test was lower than for the Caucasian sample. The mean score 
diff rence was significant at the .01 level, and was about one-half 
standard deviation in size. 

Work Sample : The mean scores of tn^ Black sample on all three 

tasks in the Work Sample were lower than for the Caucasian sample. The 

f 

meaur-dif f.erence_s were significant at the .05 level, a nd w ere about one- 
third standard deviation in size. On the Work Sample Composite, the 
effect of standardizing and combining the scores was to make the Composite 
score mean difference significant at the .01 level. 
Mexican-American - Caucasian Comparisons 

Supervisors ^ Ratings : On one scale. Perseverance, the mean for the 
Mexican-American sample was higher than that of the Caucasian sample, the 
difference significant at the .05 level* The mean rating of the Mexican- 
Americans was also higher on the Dexterity scale, but not significantly 
so. The Caucasian mean rating for Learning Ability was* slightly higher, 
but this and other differences were trivial. 

Job Knowledge Test : On the Job Knowledge Test, the Caucasians 

scored higher on the average than the Mexican-Americans. The mean score 
\ 
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dif f erence ^was'signif icant at the .01 level, and was almost one standard • 

*♦ \ , • , -\ \ * • » • 

deviation in size. The MexicaiT-Amerrcans also stored', almost one-half 

standard \Jeviation below the Black sample on this test. At least part 

of this differenced^ probably be attributed to the lang^uage handicap 

not^d on the Vocabulary Test scores in the prediction batteries. 

Work Sample ; On-one*of the Work ^Sample tasks. Logical Contjouring, 

the mean score of the Mexican-rAmericans was significantly lower/ (at the ^ 

.01 level) than for the Caucasians.' The mean dif f ererfde' was ^most one^ 

standard deviation in size. The Cartographic Technicians at San Antonio, 

the majoflty of whom were Mexican-Americans, .indicated that the Logical 

Contouring task was less familiar to them. On the ocher/two -tasks, whij-e 

the Mexican-American sample means were slightly lower than the Caucasian . 

sample, means, the mean difference, was not significant. On the Work Sample 



Composite, the effect of standardizing a*nd combi.iing the ccores was to 
make'the mean.^di^f f erence significaifc' at; the .0^1 level. >' 

Figure "lV-2 shows graphically the differenced" between Caucasian mean 
scores and Black and Mexican-American mean scores for aptitude tests (see 
Chapter III). and se;Lected criterion measures for Cartographic Technicians 
- TOPOCOM (supervisors' ratings of Learning Ability, Job Knowledge, and 
Overall effectiveness; Job Knowledge Test; and Work Sample Composite). 
The differences between the means are plotted in terms of standard deviation 
units, showing the Caucasian mean as point of departure. 

Table IV-4 shows the intercorrelations of the criterion measures for 
the Cartographic Technicians (TOPOCOM). The intercorrelations for the 
ethnic group samples' are shown on the same table for comparison. It will 
be seen that the rating scales, including the overall rating, are higHly 
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intercorrelated for all three samples, ranging from .59 to .93. The 
corre]ations>^f the ratings with the Job Knowledge Test 'range from .13 
to .51 across the three ethnic groups. The correlations of the super- 
visors' ratings on the Job Knowledge scale with the Job Knowledge Test 
stfor^ ar-e .39, .37, and .38 for the respective ethnic groups, remark- 
ably consistent, and sonjewhat higher than validity coefficients usually 
found where supervisors* ratings are the criterion. The intercorre- 
lations of the ratings and the Work Sample tasks are low, in general, 
ranging from -.02 to .49 across the ethnic groups. The correlations 
of the Job Knowledge Test scores with \]Xe Work Sanlple tasks range from 
.22 to .44, showing some overfap cf knowledge and performance criteria. 

Examination of the intercorrelations for the three ethnic groups 
shows that the intercorrelations of the Mexican-American sample tend to 
be slightly higher than for the Black and Caucasian samples, as^seen in 
Che following summary. 

s 

Range of Correlations 



Mexican- 
Blacks Americans . Caucasians 

Ratings with Ratings .61 - .90 ' .68 - .93 .59 - .89 

Ratings with JKT . ~ '^^ '^^ " '^^ ~ 

Ratings with -Work , Sample j^^^ _ _ .06 - .23 

Restitution 

Ratings with Work Sample _ _ ^33 ^02'- .14 

Logical Contouring 

Ratings with Work Sample ^ _ ^,^9 _ 

Pull-up 

One reason for the higher intercorrelations of the criterion mea- 
sures for the Mexican-American saiTiple may be, the relative homogeneity 
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of the sample. All but one or two subjects in this sample were from 

one installation. It was possible to obtain at least two ratings for 

each subject, and in some cases three or four. These ratings were 

averaged, probably contributing to higher reliability. Later analyses 

may throw "more light on these differences. 

Ethnic Group Com parisons (Coast & Geodetic Survey) 
. ^ 

The sols criterion measure used for this sample was Supervisors' 
Ratings. Table IV-5 shows the means an4. standard deviations for the 
Black and Caucasian samples. 

Black - Caucasian Comparisons 

The Black sample received lower mean ratings on all scales, in- 
eluding the Overall Rating. Onone scale. Job Knowledge, the difference 
was significant at the .01 level. On two scales. Accuracy and Learning 
'Ability, and on the Overall Rating, the difference was- significant at 
the .05 level. Differences ranged from one-half to two-thirds standard 
deviation j,n size.; Differences, on the other scales were not significant! 

- ! 

•s. 

large. ' 

Figure IV-3 shows graphical/Tl^ the differences between Caucasian and 
Black mean scores_fbr aptitude tests (see Chapter III) and selected cri- 
teripn measures for Cartographic Technicians - Coast & Geodesic Survey 
(supervisors' ratings of Learning Ability, Job Knowledge, and Overall 
effectiveness » Tba differences between the means are plotted in term|j 
of .standard deviation units, showing the Caucasian mean as point of 
departure. 

Tab3e IV-6 shows the intercorrelations of the ratings for the two 
ethnic groups in the Coast & Geodetic Survey samples. It can be seen 
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that the intercorrelations for the BJack sample tend to be highfer on 

the whole than for the Caucasian sample. It appears that in ratings of 

P 

Caucasians there was more^ differentiation on the basis of the behavior 

4 

defined by the separata scales, wfiereas supervisors tended' to rate the 
Blacks on a common factor across scales. 
Inventory Management Specialists 

Two types of criterion measures were used in this study: Super- 
visors ' Ratings (nine scales describing aspects of performance on the 
job and a rating of overall effectiveness) and a ^prk Sample. 

Rating Scales ^ 

TheTating scales are described below in behavioral terms relating 
to job performance: 

Organization 

Ability to organize work, plan ahead, and issign priorities 
Interest 

^ -Interest- and pride- in" work- and initiative in improving 

performance 

Learning Ability 

Ability to absorb and understand new materials and techniques 
Communication . 



-^bjULity to communicate effectively in person and in writing 



Technical Knowledge (Job -jGiowlecige):: - 

Knowledge and comprehension of the body of technical infor- 
mation and procedures surrounding the job 

Stability/Adaptability 

Ability to adapt tu ucw procedures and conditions; stability 
in emergencies 
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Dependability 

« 

Reliability in carrying out assignments with minimum 
supervision 

0 

Judgment 

Ability to use practical judgmeflt in making decisions 
and solving problems \ 

" tSoopera'tion 

Ability to initiate and maintain effective inter- 
personal relationships with others on the job 

C h^p.rall Rating 

Overall effectiveness as an Inventory Management 
Specialist 

Work Sample 

As described more fully in Chapter JI, the Work Sample is a simu 

lation of the job of an Inventory Management Specialist. In the test 

> 

the subject is presented with a set of problems like those usually en 
countered in the job. The scoring dimensions are described' below: 
Takes Leading Action 

Takes action toward solving the problem (asks for 
information, decision, advice';, initiates necessary 
documents) 

Uses New Procedures 

Uses new procedures in taking action on-problenis » 
(refers to or uses background information, pro- 
cedures and codes, forms provided in Work Sample) 

Shows Inventory Management Knowledge 

Shows inventory management knowledge (drafts mes- 
sages, shows understanding of MILSTRIP, takes 
actions not specific to new setting based on 
present knowledge of inventory management) 

Analyzes Problems 

Analyzes problems (takes into consideration 



89 



-92- 

0 

background information and information in other 
problems to (analyze situation, make recommendations, 
or arrive at decisions beyond whkt is specifically 
asked for in problem) ' 

«^ 

Organizes Systematically 

Organizes systematically (recognizes priorities) 

7 

r 

Maintains Control 

Maintains control X^akes prov^ion-vf or follow-up; 
records information on inventory records) 

Follows Directions ^ 

Follows directions* '(dpes what is a^ed in problem) 

Productivity 

Amount of work accomplished (number of usual actions 
taken, number of unusual actions taken, number of 
problems worked on) 



Quality of Actions 



Quality-of— AG4^ions— taken~-(-f--rom^coring-^key-deVeloped 
.from pooled judgments of appropriate and inappropriate 
actions) 

Rating of Overall Performance 

4 

Rating of overall performance (scorer's judgment on 
a 7-point scale of how effective subject Would be in 
the assumed* role, based on initial handling of a 
sample of the work) 



Ethnic Group Comparisons 

Tables IV-7 and IV-8 show the means and standard deviations for 
the Supervisors^ Ratings apd the Work;.skmple for -Inventory ^lana^ement 
Specialists' by ethnic group. 

Black - Caucasian Comparisons 

Supe rvisors' Ratings : The mcfaiTsuprervisors' rating for the black 
sample was significantly lower (at the .05 level) on only ope scale 
(Technical Knowledge). The^ size of the/differences was less than one- 
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third standard deviation. Caucasians received very slightly higher 
mean ratings on all other scales with the exception of the Cooperation 
scale, where the mean rating for the Black sample was slightly higher 
than for the Caucasian sample. 

Work Sample ; For the Work Sample, all mean scores of the Blpck 
sample were lower than for the Caucasian sample. The mean score dif- 
ferences were significant at the .01 level. 'The size of the 
differences was about two-thirds standard deviation on most scores and 
almost one standard di^yiation on the Quality of Actions score. The 
standard deviations of the Black sample were somewhat smaller on all 
, scores than for the Caucasian sample; indicating less dispersion of 
scores. 

Mexican-American - Caucasian Comparisons 

Supervi^OTs* Ratings: The mean^sAx pervisors '^ra ting-lor _the 

Mexican-American sample was significantly lower (at the .05 level) 
on only one of the rating scales (Communication). All other mean 
differences were trivial, in some cases favoring the Mexican-American 
sample. 

Work Sample : None of the mean score diffex^cW between the 
Mexican-American 'sample and the Caucasian sample was significanc. 
^ On some of the scoring dimensio*is, the sipall mean differences favored 
the Mexican- Americai^fe ; on others,^ the Caucasians. 

Figure IV-4 shiws graphically the differences betwee^^^aucasian 

mean scores and Black and Mexican-Amer ican mean scores on aptitude^ 

tests (see Chapter III) and selected criterion measures for Inventory 

Management Specialists (supervisors 'i^atings of Learning Ability, Job 
f 
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Knowledge, and Overall effectiveness; and Work Sample Overall Per- 
formance). The differences between the means are plotted in ter-ns 
of standard deviation units, showing the Caucasian point of 

departure. 

Table IV~9 shows the intercorrelations of the criterion mea- 
sures by ethnic group. 

^Ratings with Ratings Comparison s 



The intercorrelations of the rating scales range from .46 to 
►84 for the Black sample, from .34 tD .83 for the Mexican-American 
sample-, and from .41 to .85 for the Caucasian sample, perhaps again 
indicating a tendency noted before for the Blacks to be rated on a 
common factor across rating scales. In looking at a distribution 
of the correlations by ethnic group, the intercorrelations for the 
— M e xica n ^Tntir lcan s^mple^ tended to spread mote wixlely over tlie range 
than for either of the other groups. Some slight indication of the 
emphasis given the various characteristics described by the scales 
by the supervisors can be noted from the intercorrelations of the 
rating scales with Overall Rating. For the Blacks, the highert cor- 
relations of scales with Overall Rating were Organization, Interest, 
Dependability, and Judgment; the lowest correlation was with Cooper- 
ation. For the Mexican-Americans, the highest correlation with 
Overall Rating were Organization, Interest, and Stability/Adaptabil- 
ity; the lowest correlations were with^ Communication, Technical 
Knowledge, and Cooperation. For the Caucasians, the highest correl- 
ations with Overall Rating were Organization, Stability/Adaptability, 
and Judgment; the lowest was with Cooperation, All correlations 
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with Overall Rating were fairly high, ranging from .54 to .85 across 
ethnic groups, so the differences are not very meaningful. 
Ratings with Work Sample Comparisons 

In general, the intercorrelations of ratings with Work Sample were 
low, ranging from .07 to .35 for the Black sample, ftom -.07 to .42 for 
the Mexican- American sample, and from -.08 to .34 for the Caucasian 
sample. Correlations of rating scales with Work Sample Overall Rating 
for the Black sample, significant at the .01 level/, were Organization, 
Learning Ability, Technical Knowledge, Stability/Adaptability, Depend- 
ability. Judgment, and Cooperation; for the Mexican-Americans, Learning 
Ability and Communication; and for the Caucasians, all scales except 
Dependability and Cooperation^ The intercorrelations across rating 
scales with the W^rk Sample Overall Rating for th^J Black and Caucasian 
samples again may be evidence of a common factor underlying supervisors' 
ratings, but the reason the Mexican-Airi?.rican sample has different rela^ 
tionships is not understood. Later analyses may throw^more light on 

these differences. j 

Summary 

For the Medical Technicians and Inventory Management Specialists, 
the mean supervisors* ratings are only slightly different for the ethnic 

groups. The mean supervisors* ratings for the Cartographic Technicians 

1 

again are quite similar, but differences dcj' appear among the groups on 
various scales,^ suggesting that some aspects of job performance for the 
ethnic groups were seen differentially. For the Cartographic Technician 
sample, multiple ratings were obtained at some installations, perhaps 
increasing their reliability when averaged. 
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On the Job Knowledge Tests, the Black Medical Technicians and the 
Black Cartographic Technicians scored lower by about one-half standard 
deviation than the Caucasian groups. The Mexican-American Cartographic 
Technicians scored lower than the Caucasians by almost one standard 
deviation (and one-half standard deviation lower than the Black Carto- 
graphic Technicians). In the latter instance, the Mexican-American 
results on the Jpb Knowledge Test may reflect bias due 'to a language 
handicap. (Blacks scored higher on Vocabulary tests and some other 
tests involving language in both the Cartographic Technician and the 
Inventory Management Specialist studies, although Mexican-Americans 
scored higher than Blacks on rote memory, perceptual speed, number 
fluency, and tests of general reasoning.) 

On the Wore Sample for the Cartographic Technicians, the mean 
scores of the Black Cartographic Technicians were lower on all three 
tasks than the Caucasian mean scores (at about the same magnitude) . 
However, the mean scores of the Mexican-American Cartographic Techni7 
cians on two of the tasks were not different from the mean scores of 
the Caucasian sample. On the Logical Contouring task, 'the Mexican- 
Americans were one-half standard deviation lower than the Caucasians 
and one- third standard deviation lower than the Blacks, suggesting 
possible differential experience on this task among the groups. 

On the WorK Sample for the Inventory Management Specialists, the 
Black sample scored consistently lower than both the Mexican-Americans 
and the Caucasians. This Work Sample, while highly verbal in content, 
was developed around the actual work performed by Inventory Managers, 
and the language handicap of Mexican-Americans, reflected in their 
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lower scores on aptitude tests involving the use of -language (in which 
Mexican-Americans scored lower than Blacks as well as Caucasians), ap- 
peared not to be a factor. 

Of the three kinds of criterion measures, the supervisors' ratings 
appear to reflect least well the differences ^ong the ethnic groups 
shoim by aptitude tests and other criterion measures* Further examina- 
tion of rating variations and sources of rating bias will be made in 
Chapter VIII. 

Black subjects scored less well than Caucasians .on the Job Knowl- 
edge Tests in both the Medical Technician and Cartographic Technician 

'studies. The Mexican-American Cartographic Technicians appeared to do 
less well on the Job Knowledge Test than their performance on other 
measures would indicate. Whether this is attributable in part to lan- 
guage handicap is not clear. 

The Work Sample, with the exception of the Logical Contouring task 

' for the Mexican-Americans, appeared to elicit somewhat consistent results 
from the ethnic groups in the Cartographic Technician ^tudy.' On the Work 
Sample for the Inventory Management Specialist study, the Blacks performed 
consistently less well than the Mexican-^ericans and Caucasians, but they 

/ also had lower aptitude test scores in general (Wth the exception of the 
Vocabulary tests mentioned above) . 
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Criterion 



Table IV-1 

SUPERVISORS' RATINGS AND JOB KNOWLEDGE TEST SCORES 
MEANS AND STANDARD DEVIATIONS 
BY ETHNIC GROUP 
Medical Technicians 
Black 



Caucasian 



Measures 


N = 


166 

f 


N = 


290 


- 


Mean 


S.D. 


Mean 


S.D. 












Flexibility 


5.5 


1.8 


5.4' 


1.8 


OrganiEation 


5.7 


1.8 


6.0 


1.8 


Interest 


5.6 


.. 1-8 


5.9 


1.7 


> Learning Ability 


5.8^ 


1.9 


6.0 


1.8 


Job Knowledge 


5.2 


1.9 


5.3 


1.7 


Technique 


5.9 


1.7 ' 


6.0 


1.7 


Low Need for Supervision 


5.7* 


2.0 


6.1 


1.9 


Coiranunication 


5.5 


1.8 


5.8 


1.8 


Overall Rating 


5.7 


1.9 


5.9 


2.0 


Job Knowledge Test --^^ 


31.4** 


10.4 


35.7 


9.4 



^Significantly different 

^ f rom Caucasian mean at .05 level 



**Signif icantly different , 

from Caucasian mean at .01 level 
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Table IV-2 



INTERCpRRELATIONS' OF CRITERION- MEASURES BY ETHNIC GROUP 
(first line Black sample, second line Caucasian sample) 



Rating Scales . 



Medical Technicians 



R 



4 

7 



R R 

% 6 



R 



Job ' 
Knowledg 
Test 



R Flexibility 1.00 

^ * i.bo 



R^ * Organization 



R^ Interest: 



R^ Learning Ability 



Rc Job Knowledge 



Rg Technique 



R^. Low Need for Supervision 



Rq Communication 
o 



. 75 ■ 


.80 


.77 


.76 


.70 


.65 


.77 


.50 f 


.61 


.73 


.55^^ 


.66 


. 65 


.51 


•69 




.74 


.77 


.12- 


.76 


.78 


.68 


.78 


-.45 


.70 


.71 




.74 


;-76 


.61 


.74 


:.27 




.73 


.70 


.73 


.69 


.67 


.74 


.■38 




.70 


-.63 


'.64 




.61 


.73 


.22: 






.84 


'V82 


..79 


.75 


.80 


;»58 






.72 


.7.5 


.71 


.66 


.74. 


.35. 








.79 


.75' 


.78 


.77 


:58^ 








.69 


..73 - 


.•70 


.72 


- .35 . 










.79 


.72 


.81 


.49 










.73 


.65 


.75 


■.23 










* 


.73 


.79 


.43 












.71 


.77 


. ;27 . 














»70 


^50 














.65 


* .27 



Rg Overall Rating 



V45 
.,25 



Job Knowledge Test 



Signif 4cance 

Black 
sample 

. Caucasian 
sample 



^r 

\ 



il6' = sifenif icanr at 
* .21 =\significant at 

.11 significant at. 
.15 = significant at 



.01 
'.05 

.01 
.05 



level 
leveJL 



level / 
level / 



• A ■■ ■ 

Table IV-3 

' , ■ ' ■ . < . 

, , SUPERVISORS' RATINGS, JOB Kn6wLEDGE TEST AND WORK SAMPLE SCORES' 
. • - flEANS AND STANDARD DEVIATIONS BY ETHNIC GROUP 



Cartographic TecWi'cians (TOPOCOM Sample) 





Biack 
N=10i • 


■ Mexi c an- Amer i can 
N=101 • 


llaucisb 
• ■ ■ N^] 


ah 

L'' 


Ra ting 'Scales 


Mean 


S.D. 


Mean. 


S.D".' ' 


Mean 


S.D. 


• ^Accuracy ' 


5.4* 


-1.8 


5.-8 ■ " 


i.7 


' 5.9. 


1.7 


\H[ritefest i 


5.. 3** ■ 


1.7 


5.9- 

t * a - 


■1.7 ' 




i:6. 


'Learning: Ability . 


5.4* 


1.7 


5.6 


1.7 


' 5.8 ^' 


1.7 




5,.3. 


.iv7" 


5.6- . 


i.8" .' 


5.7 


1.8 . 


^ \^ 'dexterity * 


5.9 / : 


■ l76 


6.3 


1.7 ■ 


; 6.1 


1.6' 


. Needf for 
'Supervision 


, 5.1* 


i;.8 


6 

' 5.5 • 




5 .-6 


1.8 


Perseverance • 


6.1 


1.6 


6.6* . 


1.5 


6.2 ■' 


1.7 


; Oyer all. Rating 


5.5. 


1.9 


5.8 


1,6 


5.9 


1..7 . 


, Job. Knowledge test"" 


35.4** 


11.4 


f 30.6** 


10.0 


. 40.3 


11.6. 


"iWork. Sample 












» 


■ ,-, • ResCitution 


17.3* 


6,. 9 


. 18.8 


7;"5 


19.1' 


7.6 


LoglTcal Cbnt curing 


8:8* ■ 


3,4 


7.2** 


4.1 


9.6 ' 


2.9 


Puii-up_JL-,___- 


J.-lr.-3* 


5.4 


i 12.2 


5.3 


12.6 ' 


5.1.: 


Cdmppsite// 

" ' ■ /■ 


18.3** 


9.2 


:17.9** 


io.7 


21.6 


8.9 , 



*Signfficantiy different * 
"^f rbm^'^'Caucasian mean at .05 level ' . ' 

**Significantly different . ^ • 

from Caucasian mean at .01 level 

■./ ^ ' / ■ 

.//Obtained by standardizing and combining the raw scores for the separate tasks 
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'table 

Intercorrelations oH Criterion Measures by Ethnic Group 
Cartographic Technician^/j^TOPOCOM) 
(Black - first line; Mexican- American - second line; Caucasian - third line) 



Rating, Scales 



R^. Accuracy- 



Interest - 



R^ ^Learning' Ability ■ 



Rj^ Job Knowledge 



■ R^ Dexterity 



. .Rv Need fpn 

Supervision 



1,00 

l.jDO 



Perseyerance 



Rg Overall Rating 



Job Knowledge Test 



.7h 
.78' 



3 






o 


7 




JKT 


i. 








— — 


' .86 


.69 




.22 


.09 


.05 


.89 


.90 


'.78 


.85 


.74 


■ .90 


■ .12 


.-22 


.29 


.79 


,86" • 


.77 


.oil 


.75 , 


QQ ■ 
.OO 


• 33 




nft 
.uo 


.7h 


.72 


" .61 


.75 


.78 


. .79 


.15 


•--.01- 


-vO'2— 


.87 


.83 • 


:75 


..83 


.86 


.87 


.li7.. 


-.11 


.21- 


.77 , 


.79 


.65 


i78 


.77 


.77' 


• 3l» . 


. lU 






.87 


.75 


• 85 


762- 


.83 ' 


.-35 


.11. 


.13 ■ 




.92 


:i79 


..9X 




-.88 


.51 




.27 




.85 


.73 


..85 " 


.6h 


.83 




♦23! 


.111 






i73 


.90 


.68 


.85 


.39 


.i3 


.13 






.78 


.93 


.72 


.88 


' .37 


.17 


.21 




< 


.7h ' 


• .89 


.70 


.85 


.38 


.17 

c 


.13 








nti 
■ .7p 


Aft 
. 00 






TO 


w OP 








.81 


.68 


.85 


.38 


.09 


.38'* 








- .76 




. .80 


.27 


.17 . 


.Oh 










.73 


.86 


.311 


.1 

:10 


.11 










.73 


.89 




.16 


.2V 










.68 


.86 


.39 


.22 


..Ik. 












.76 


.13 


:.oi 


.02 ' 












.77 ■ 


.38 


■■ .06. 


■ .11 ■ 














.25 


\G6 


.02 






0 








• -.28. 


.02 . 


f 

. .11 






« 


.* 








■'.15-- 


:29 














.36 


.17 


'..07 . 
















.22 ■ 


.31 
















*.U5 


.27 
















.Ul - 


.30 



.Ih 

.38' 
.16 

'.or 

.32 

^26 ■ 

.i'8 

:lt2. 
--27 

.18 

>:38^ 

i23- 
".07 

iU9; 

•17 

.ill 

;25 

.•15 

'.15 

06 
.16; 

. .39 
.\x2 



.13' 
.22." 

rbii 

i29, 
i23* 

.'21^ 
.423 
.31: 

v2i; 

^25-i 
.'67X 

Ah. 
;365 

;64i 

.14: 
.3^ 
.19 



^56 
55 



WSj^ Restitution 



V/S- Logical 



Contouring 



WS^ Pull-up 



V/S)Corririosite 



Significance 

Black . 
& 

Kexican- 
Americah 
samples 

Caucasian 
sample 



.25 = significant at .01 level 
,20 = significant at .05 level 



,18 = significant at .01 level 
,14 = significant at .05 level 



.16 
.29 
.10 



.68 
.17 

.uo 

.•t*3 
.27 
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T^ble IV-5 
SUPERVISORS* RATINGS 



llEANS AND STANDARD -DEVIATIONS BY ETHNIGs GROUP 

■ ' * ^ ■ /'I 



*Sighif icantly- different ^ 
f r<Sn Caucas^ian mean at .05 level 

**Significantly different 

f rom Caucasian mean- at .01 level' 



Cartographic Technicians (Coast & Geodetic Survey Sample) 




• - 


DXaCK 


Caucasian 




' . ' / N^38 ' . ^ 


• N-50 




Rating. Scales. 


Mean S.p. 


Mean 


S..D. 


Accuracy 




6.i 




Interest- . 


5.5 ■ . 2V2, . 


6.1 '/ 


1.9 


Lear^^^ Ability 


5.1* 2.r. 

It 


6.1 


1.8 


Job': 'knowledge 


4.9** 2.1 ^ 


,6.0 


1.7 


Dexterity 


• 6.1 2.6 • 


'6.3 


1.8 


■ •*Nfec?d?'^dr; Supervision 


5.3 id 


„ 5.9 


1.8 


-.-^Perseverance ' 


6.0 2.3 i 


. 6.6 


1.8 


' Oyer all Rating 


5.4*' / - ' 


« 

'6.2'- 


i.6 






- 
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Table IV-6 

INTERCORRELATIONS OF SUPERVISORS* RATINGS BY ETHNIC GROUP* 
Cartographic Technicians^ (Coast & Geodetic Survey Sample) 
(Black - first line, Caucasian - second line) 



Agfeuracy 

Jnterest 

>^ 

Learning 

Job .Kiowledge 

Dexterity ; 



Need ' f or^. ^ 
.Sujpervision 



.Perseverance 



Overali Rating 



1.00 
1.00 



' 2 

.89 
.78 



.85 
.82 

.81 



h 


5 


6 


.7 


r 

• or 


...07 
-.76 


.02 

,78 


.ou 

i66 


y 

.70 


.80 *■ 
.56 


■..79. 
.70 


-^•.93 
.75 


.92' 
-.86 


.81 
.72 


.8U 
.86 


.72 
-.55 




,.86 . 


.90 


.80 
.62 




V 

1 

\ 


V83- ■ 
.72 


.73 

:h7 






.7U 

..57: 



Significance 

Black ■ ^ 
. sample 

Caucasian 



.42 = 
.32 = 



significant at .01 level 
significant at' .05 level 



."29 = jsignificaAt at .01 level 
."SV = significant at .05 level 
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Table IV-7 


i 








MEANS AND STANDARD DEVIATIONS FOR SUPERVISORS' 


RATINGS 










BYETfiNIC GROUP " 




• 






• 


Inventory Management Specialists 




• 




• * - 




• Black Mexican-American 


Caucasian 








^ . • N=114' * . N=74. 


N-205 








Rating Scales * 
Organization 


Mean S.Dv • Mean S.D. 
, 6.0 1.9. 6?4 .1.7. 


^ Mean, 
6.2 


S.D. ' ■ 
1.9 


• * 


' • ♦ *^ 


^Ihtetest 


. „6.0 ,1,9. 6.3 .1.8 


6.x 


1.8 






^ Learning ^ / 
Ability 


6.6 *.■ 1.9 ' . .. 6.1 .1.8 

• 


6.1 


1.9 






Communication / 

'.Technical . ^ 
' * ^Knowledge ? 


5.8" 1.7, 5.5* _ 1 . 9 ^ 
5. '8** - 1.9 6.3 1,6 


„^ .6.0 ■ 
6.3 . 


1.8 
1.8-' 






^ '^Stability/ ..^ 
* AdapjiabiliEy. 

Dependability . 


6to ^ ; 1.9 '6.1 * ■ ■ 1.6 
' 6\3 2.0 6.6 ■ 1.6 


6.0 
6.5. 


1.9 
1.8 


,V 
^. 




Judgment 


6.-2 -1.8 . 6f3-, 1.6 


' "6.3 " . 


li8 






Cooperation 


- . 7.'2 • 1.6* ' ' 6.9 1.6 




i.5 






Overaii Rating ^ 

" 


6.2 - . 1.7 o . ■ •6.4* 1.7' 


6.3 


1.7 






*^.»»'_ 

<^*Si^h^f:^can^lyf differs 

from -Cajjcasian mean at .05 level 




\ • 


»^ 

■1 X z 












J 



























*■ . Table IV-8 • . ■ 

MEANS AND STANDARD DEVIATIONS OF WORK SAMPLE DATA 
BY ETHNIC GROUP 
Inventory Management Specialists 
Black 

N=100 ..N=59 



Work Sample 
Dimensions 

Takes Leading 
Action 

UsesivNew 

-T Procedures 



, Shows I.; M. 
i^dwlldge 



Analyzes^ 
Problems 

'Organizes 

■Systematically 

^!aihia^n3 



;^jC6nt:rols 



Mexican-American-- 

4 



Follows 
' Directions 

Ptfoductivity 

Quality of 
Actions 

Rating 'of 
Overall • 
Perfiormance 



Mean 
2.9** 

7.4** 

3.6** 

1.5** 

4.2** 

1.8** 

3.9** 
44.6** 
10.2** 



S.D.. 
2.3 

3.8 I 

2.5 

1.4 

3.1 

• 1^6 

2.1 - 
22.3 
5.2 



6.6** 3. A 



. Mean 
4.5 

10.0 

5.1 

2.2 

6.0 

3.2 

5.5 
58.8 
14.0 

9.0 



S.D. 
2.9 

5.1 

3.1 

1.9 



3.6 



2.3 

3.1 
33.4 
6.4 

4.1 



Caucasian 
N=171 



Mean 
4.4 

10.2 

5.1 

2.6 

6.2 

2.6 

5.6 
57.3 
14.5 

8.9 



S.D.. 
2.8 

5.2 

3..0 

' 2.3 

4.6 

2'. 2 

3.2 

34.9 
7.0 

4.5 



**Srgnificantly different 

from Caucasian mean at .01 level 
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^ ^ , ■ . * - ' . Chapter V . ^ • 

/ V Validity of/ Aptitude^-^Tests for Different Ethnic Groups - > 

— "THls chapFer deals "WitTTnEhe valid bf th6 apfitude t^sts des,cribed 
In Chapter as measured by the degree - of their relationship with the 

criteria .of job performance described in Chapter IV. The questions to be 
*l explored are: 1) *afe the selected aptitude tests valid^ predictors of job 
.performance for the occupations studiecf; 2) do the validity coefficients 
obtained diff^er fpr the separate 'ethnic- groups; .and 3) do the validij:y 
, * c6eff,icierits differ for the various criterion measures?* In the following 

r, >i'* ■ '' ^ " - " ■" ' 

, 'sections, the relationships between aptitude test scores and 1) 'supers 
visors* ratingo, 2) ipb knowledge test scores, and 3)* wor samp le* measures 

A wiM be difecus^srfe^^y <ocGUpat'ion and by ethnic group . ^ 

■ ' ^ ^ - 'l '^ . ' • ■ • 

alidity of. Aptitude Tests Against Supervisors * Ratings 

^ - . ^ ' ^ ' • > 

• In this sectipn, corfelatidnS of aptitude tests \3rith supervisor s-* rat- 

ings on ^two rating scales used for each occupation -^studied are presented. 

Ttjese" ra^tihg scales are: 1) Learning Ability (rating of ability to learn 

on the job) and 2) Overall (rating of overall effectiveness on the job). 

The ^^irst of these was chosen siace ability to learn on the job would 

■K^ ~ ^appear^, to 'be -closely ^related to the purpose of most of the aptitude tests, 

''Correlations of the aptitude testp wi-tft the ratings of overaii^jerforjnance 
V ' ^ ' c - . . ' . ' / *" . 

^- will give an indication of 'the extent tcf^whicK the abilities measured by 

fhe ;tests Prelate to ^ffectUve joB performanc<e« • • ' 

- ' ' . " / ' . ' • * *" 

* Table V-l shows the correlations of aptitude tests with supervisors* 

f * ^^^^^ ^ " , ' 

•^ratings foj: ..Medical Technicians by^ ethnic group. It can be seen ^that (1) 



' 11. 



Val.lidity coefficients for aptitude tests^ against supervisors* ratings, 
on all scales for the three occupations studied are shown in Appendix 
• Tables y-A, V-B,'* v/c/ana V-^D. • . . ' . ' ^- 
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test validities are generally higher when tha criterion is. learning abii- 
ity than when it is overall performance and ^(2) test validities are higher 
for Blacks than for Caucasians on tests measuring traditional academic 
areas (verbal and mathematical skills), but higher for Caucasians than for 
Blacks on tests of spatial perception, finger dexterity, and irote memory. 
None .of- the correlations for Blacks is significantly different from the 
corresponding, coefficient, for Caucasians. ' 

- Table V-2 shows fhe correlations betveea aptitude tests and super- „ 

\ # ^ " . . , ' ' 

.visors^ ratings for Cartographic Technicians at TOPOCOM installations by 

ethnic group. Again, the highest coefficients are those against tfie ratr 

icigs on learning ability., fhe vocabulary tests app,ear to have little or 

no validity for Caucasians or Mexican-Americans. While the coefficients 

for Blacks are higher and consistently positive, they do not reach t^^^ 

ieveL of significance. In contrast, the tests of spatial and perceptual 

-abilities, arithmetic, and reasoning haVe significantly high positive 

coefificients for all three groups against the learning ability ratings, 

^fiile against the overall ratings these coefficients are consistently , . 

positive, although a number do not reach the significance level. In ho 

instance is a minority group coefficient significantly different from the 

corretsponding coefficient for Caucasians. 

* i 

'Table V-3 shows the correlations between the aptitude tests and the 

• It ' . . 

fev^oj*rating,,6cales for Cartographic Technicians (Coast and GeodeCic Survey) 

The patterns of ^relationships are quitie similar ,to those^ found for the 

_* • * " ■ » • 

^TOPOCOM technicians, although the sizes of the correlation coefficients 

are," in most instances, x,a little highef; In every instance except one, 

.there is no significant difference between the coefficients for the 



Caucasian group and the .coefficients for the two minority -groups. The \ » 
exception is the correlation between thfe Extended Range Vocabulary .Test 
and' Learning Ability rating. For Caucasians, the coefficient is .35, 
for Blacks, -.11. The difference in coefficients is significanfe at the ' - 
.01 revel. ^ . ^ 

_ -Table V~4 shows the correlations between aptitude tests and super- 
visors' rat^gs 'for Inventory Management Sl)eciaiist:s by ethnic group. 
Again,, correlations of aptitude tests with the Learning Ability rating 
are jgeher ally higher than .correlations with the Overall rating, but the 
.differences are not very large for this occupatioh. Although.41^ ^b? 
coefficients\are positive, the coeffidients for the two vocatjulary tesVs; 
ahd^for the Object-Number- (short-term memory) Test are quite low. While 
miich^of the work of inventory Management- Specialists is verbal* in nature, „ 
-it appears that beyond .some level, additional- increments of verbal ability 

' . ' ' ' ^ - ' * . ^ * ' 

" oir short- terra memory do not affect supervisors' evaluations of job perfor- 
finance. The other aptitude measures have substantial relationships with . 
supervisors' . evaiuatio^. . In general, the relationships are higher for 
Blacks and Mexican-Americans'" than for Caucasians.' 
Vaiidity of Aptitude Tests Against Job Knowledge Test Scores 

iFor two of the occupations studied^ Medical Technician and Carto- 
graphic Technician, it' was possiblie to obtain or construct a paper-and- 
pencil test of job knowledge. In both of these fields,^' this kind of 
knowleidge iz considered essential to job performance, and can be acquired 
on the job as well as through education or other ttaining. A justification 

for using a written test format is, the fact that use of ^written materials 

* > ■ « ^ 

and .communication in writing is an^"*esSential part of the job. 
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Table V-5 shows the correlations between, aptitude tests and Job Knbwl-: 

; . ; ^ . . ^ * 

edge Test scores for Medical^, Technicians • All of the coefficients are 
.positive, and all coefficients for the Black sample are higher than the ' 
corresponding coefficients for .the Caucasian sample However, none^ of the 



differences between pairs of coefficients reaches the level of significance. 

«^r' ' ^ . " \ ' ' ' ■ / . - ~ - ..-^ ' 

» Table V-6 shows the correlations between./aptitude tests and Job Knowl-? 
edge. Test scores ?for Cartographic Technicians at TOPOCOM installations. 
All but one of the coefficients are positive. Two of the coefficients for. 
the Caucasian sample are significantly di ehtj at the .05 level, from 
thje^'correspondihg coef f icient ^ for one of the* minority samples ♦ One of these 

' iriC^ ' ' " t.- " ' ''^ ' / - ■ ^ • " ^ * 

instances is for the Coordination Test^. where the coefficient for«* Caucasian 
technicians is .21 and the coefficient for Black technicians is -.06. The, 

other is for the Arithmetic'Test, where the coefficient is .43 for Caucasian 

t , . • 

technicians arid ;62 for Black ^technicians. ^ 

^ . ^- ■ 

Validity^ of Aptitude Tests Against Work Sample Measures ' ^ 

Work sample problems, as described in Chapter II, were developed for , 

the Cartographic Technician and Inventory Management Sp.ecialist occupations. 
Table V-7 shows the correlations between aptitude tests and the Work 

Sample Composite score- (cpmpostte of ^standard ^ores on three individual 

problems) for Cartographic Technicians at TOEOCOM installations. All but 

one of the coefficients afe^positive. The Object-Number Test, howevei^^ has' 

» .' ' . 

very low validity for all three groups,, and the two vocabulary tests have 

essentially ^zero validity for the Mexican-American group. , For the first 

■ ^ / - » •- . 

vocabulary test, the coefficients for Caucasian and Mexican-American sampler 
are significantly different from each other at the .05 level. 

Table ''v-8 shows the correlations between aptitude tests and the Work 



.Sample oyeraU. score for Inventory ^Management Specialists. All of these 
doefcrfcients are positive. However, the Object-Number Test appears to 
haye- low validities for all three groups ♦ For two of the tests,, Extended; 
.Ra^ge» Vocabulary and Necessary Arithmetic Operations, the coefficients for. 
^bhe/Caucasiaii and Mexican-American groups are significantly different .from 
eadh other at the .05 lavel.A ~ ! , 

rcdmpafison of Validities Across Different Criteria i» 

tor comparison .of validities across different criterion measures, cor- 
relation coefficients- between aptitude tests and selected criterion measures 
/ftave; been plotted by .occupation . and ethnic gro The points haye been . 
\^joined^' to form what are essentially; validity profiles. ; 

/.Figure, .V-l shows prb'files of validity coefficients for Medicai Techni- , 



.ciaris for all aptitude tests against the Learning Ability rating and Job 
iOiowiedge Test scores for the Black and Caucasian samples. It, is readily 
app^reiii tli'at the patterns are similar, with, only small differences between 
the. two groups dt most points. The Black line lies above the jCaucasian -line 
at a .nuftber of points on. the Leirning^ Ability graph and at ali points on the 
JobJKnbwledge Test graph, indicating higher test validities for the Black 
/group' wherever this demurs.* In general, ihowever, where a test is valid *foi: 
one group it is also valid for the other .group", and validities for both 
types of criterion measures are comparable in degree. * 

- .Figure^ V-2 .shows.-prof iles ^of -.validity, -coeff icients for Cartographic ^ 
Technicians (TOPOCOM) for all aptitude teSts against the Learning Ability 
ratings Job Knowledge Test scores, and Work Sample Composite scores for the 
Black,^ Mexican-American, ^nd Caucasian samples. * Here, again, .the patterns 
, fare similar for all three ''groups, with, few' large differences. ^ As pointed^ 
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diit. in previous se.ctions, validities for Blacks and Mexican-American^ are 
higher, than for Caucasians in many instanc€fs^ and tests yaliiJ for one grpup^ 
aire also valid ,fj6r the other groups, wTth" few exceptions, -The differences 
in test yaiidity; across- criterion measures may be inter;^eted in view of \ 
the aspects of jo.b p.erformance bieing.measurfed.'^ For ^xaMple, the vocabulary 
tests iiave'highet validiti^ when the criterion, is^ the Job Knowledge Test ^ 
than when the Learning Ability rating or; WorK Sample Composite score are 
the criteria. A measure of rote memory (Object^l^umber) has moire relation- 
ship w4.t?h the teaming Ability rating and the Job Knowliedge than with^^ 
the* W6rlc S^ple./Cpmposite score.* . . . .. 

Figurfe V-3 .shows profiles, of validity^ coefficients /for Cartographic 
Technicians (Coast |ind Geodetic Survey) for all aptitude tests against 
the Learning Ability rating and^ Overall Performance rati'ng for the Black 

' . ^ ^ • * y - ' 

and Caucasian samples (supervisors* ^ratings were the only criteria for 
.this group). The larger differences in validities between the two^ ethnic . 
samples pointed out ia i previous section are easily seen in this figure, 
in general, validities for the Black sample aire higher than for the - 
Caucasian sample.* 

' . • . *" 

Figure V-4 shows profiles of validity coefficients for Inventory 

Management Special'ists for all apTpitude tests against the Learn^.ng Ability^ 

rating, and the Work Sample Overall score for Black, Mexican-American, and 



Caucasian samples. ~The^ validitjr profi^les for the three samples are very 
similar for the Learning Ability rating. For Lhe W9rk Sample Ov^erall 
score, the Black and Caucasian lines follow each other closely, while the 
Mexican-American profile reflects the higher validities obtainJ^in general 
for this sample. .The same generalizations may be made, however, that tests 



valid -for one group fend to be valid for another and that validities are 
geriefalizable across different t^pes of criteria/^ 
. * ' ' ' *T ' \ :Summar^, » . .~ 
in the studies of 'Medical Technicians^, Cartographic Technicians, and 
Inventory Management -Spec iali tests, selected after careful 

study o£ the job; were 'correlated with tlitee different kinds bf> criteria: 
Supervisors* Ratings, scores on^'Job Knowledge tests,] and scores on Work 
^Sample tasks. These correlations: were compute ^separatel]^.^ for Black,,, 
zMexicahrAmerican,* and C^^ * 

^ In a few instances, particular tests .showed jittle or no Validity for 

------ ; " ' ■•i . > M 

one' or more ethnic .groups. However most validities were stibsf "^ntially 
positive for all ethnic grpups, f or lall criterion measures. The degree 
of relationship was essentially the same for all threfe ethnic groups. Out 
of 229 comparisons be.tween correlation'' coefficients for the Caucasian 
'sample and the. corre'sponding coef f icients/for the minority groups on the' 
\tabies in this ChapteA one such difference, was significant at the .01 
level, and six were significant at th6 .05 level. This is fewer than would 
be expected by chance. In five of these seven instances,' the higher 
coefficient was for the minority group. . ' ' * 



lie 



/ 



Correlations Between Aptitude Tests and Supervisors^ Ratings 
' for MedicaKi^echnicians" by Ethnic Group 



"0 



lest 



Subtraction & 
Muitigiicatidn 



Vocabulary 
Hidden 



. / 

/ 



Necessary 
^^rithmetic 
Operations 



" P.fn^ 
-Dixterlty 

/ 

Number 

Comparison 

^ . _ ^ _ _ 

Gestalt 
Completion 

Picture 
Number 

- Paper 
Folding 



Learning Abilit]^ 



Rating 



Black 

/Nc:168 



•/ 



' .36** 
.20** 
.02 

. 20** 

;^ -.26** 

• 18* 
'.06 
■' .21** 



Caucasian 
■ N=285 

.07 
.12* 

. 30** 

.26** 
• . 16** 
.19** , 
.20** 
.28** 



Overall Performance 
Rating 



Black 
N=168 

. 29**\ 

.09; 

.02 

.17* 

.10 
,17* 
.05 ' 
-.0'2 
.08 



>~Caucaslan 
■ N=285 

. .13* 
:06 
- .05. 

.16** . 

.12'* 



.04 



.11 



.15** 



.11 



* significant at .05 level 
**- significant" at .01 level 
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Correlations Between- Aptitude Tests and Supervisors' Ratings ^ 
'fo? Cartographic Technicians by Ethnic Group (TOPOCOft -Sample) 



Learning Ability Rating 



Overall Performance Rating 





Black 


nex J. can- 
American- 


Caucasian 


Black 


Mexicari" 
American 


Caucasia 


Test 


N=101 . 


N=99 


v.. 

N=240 * 


N=101 . 


N=y9 




Coordination 


.15 


• 17 


.21** 


n/. 






Hidden i'igures 


.29** 


.41** 


. 25** 


.21* 


.29** 


.21** 


Vocabulary 


.17 " 


.01 . 


.03 


^.19 


-.02 


.01 


Otject-Nuiaber 


.. .21* 


• 12 


-.04 


.19 


.01 


• uz 


*Card Rotations 


.28** 


.19 ^ 


.31** . 


.16 


- .04 


.26** 


CS Arithmetic t 


.42** 


.34* 


.25** 


.31** 


.21* 


.24** 


Map'-Planairig ■ 


.33** 


.39** 


.40** 


.zA**^ 






Surface ' 
Deyeiopment 






• JH **** 


28** 


.21* 


. .28** 


Maze-Tracing 
Speed 


.20* 


.33>** 


.32** ■ 


.14 


•15 


.27** 


- Following Oral 
Directions ^ . 


.32** 


.32** 


.33** 


.18 


.15 


.•25** 


Identical ^ 
Pictures 


.33** 

* • 


.26** 


.20** 


.21* 


.18 


.14* 


Extended Range 
Vocabulary 


• .16 


.07 


-.05 * 


.17 


.03. 


-.07 


Necessary 
Jlflthmetic 


. .32** 


.36** 


.29** 

■ 


.25** 


.22* 


.19** 



Operations 



* •significant at .05 level 
** Ifegnlficafit at,. 01 level 
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Table V-3 - • 



Correlations Between SVptitude Tests' and Supervisors'^ Ratings 
for 'Cartographic technicians by B^thnic Group (Coast and "Geodetic Sur^vey) 



Test' 

iCoqfSiriatlon 
HiSden Figures 
Vocabulary 
Object-Number 
Card Rotation 
CS Arithmetic ' 

'Map Planning , 

SI 

Siif f ac'e , 
Development 

Maze Tracing 

*■ 

Following Oral 
Directions 

Identical 
Pictures 

Extended Range 
Vocabulary: 

Necessary 
Arithmetic 
Operationis . 



Learning Ability 
Rating 



Black 

N=r38 



;40* . 
.28' , 
.44** 
.51** 
.61** • 
.44** ' 

.53** • 

.21 

.43** 

.51** * 

.35* 

.38* 



Caucasian 
N«50 

.12 

.51'*** 
.02 ■ 
.09 , 
^39** 
. 48** - 
.45** 

.47** 

.45** 

.52** 

.52** 

-.11 

. 51** 



Overall Performance- 
Rating 



B^ack 
N=38 

.-.13 
. 30 • 
.09 • 
.35* 
.37* 
.51** 
.31 • 

.38* 

.11 

.32* ' 

.44** 

.21 

.33* 



Caucasian 
.N=50 

.00 

.42** 

.07 [ 

.07 

.30* 

.44** * 

.32* 

.40** 

. 33* '■ 

' .46** 



. 36'**- 



-.01 



.48** 



* significant at .05. level 
** sigriificant;at .01 It/el 

m ■ ■ : 
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Table.vU ' . ^ 

Correlations .Between Aptitude Tests and Supervisors' Ratings 
fby inventory Managers" by. Ethnic Group 

* . . , 

Learning ^Ability Rating Overall Performance Rating 



Test 

Number • Comparison 

Hidden Figures 

Vocabulary 

Object-Number . 

Letter Sets 

Nohsehse 
^Syllogisms 

Subtraction & 
Multiplication 

Extended Range 
Vocabulary 

Necessary 
Arithmetic 
Ojperations . 

Following Oral 
Directions 

Inference 

FSEE- (VA & QR) 



Black 
N=112 


Mexican- 
American - 

N=72 


Caucasian 
N=19t ' 


Black 
• 'Nt.112 


Mexican- 
American 

N=72 


Caucasi^ 
. N=191 


. 34.** 


.33** 


' .3,4** 


N 

■ . 2'9** 


.10 


.28** 


• 37** 


.32** 


.08'* 


.30**' 


-.25* 


.10 


.10 


.15 • 


.13 


.14 


.04 


.12 ■ 


.03 


" . .06 


.06 


.04 ■ 


.00 


.07 


.31** 


, .24* 


. 25** 


. 32** 






.26** 


.40** 


.13 , 


.17 


.33** 


• 10 


.35** 


.36** 


.31** 


.39** 


.25*. 


.32** 


.04^ 


.15 


.18* 


.03 


.10' 


.14. 


.36** 


.39** 


.28** 


.34** 


..25* 


.25** 


.35** 


.34** 


.18* 


.22* 


.25* 


.13 


.30** 


.22 


.21*^ 


.23* 


.13 


.20** 


.30** 


.32** 


. ..28** 


.24* 


.24* 


.22** 



* significant at .05 level 
** significant at .01 level 
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Table V-5 



Correlations Between* Aptitude Tests and Job Knowledge. Test, 
for Medical Technicians by Ethnic Group' * • ■* 



Test 



Black 
Nn68 



Caucasian 



Subtraction & 
Multiplication 



Vocaburary^ 

Hidden 
Fi^Ces^ 

Necessarjy^ 

Arithmetic 

Operations 

Pin- 
Dexterity . 

Number • 
Comparison 

Gestalt 
Completion 

Picture. 
Number 

Paper. 
Folding 



.34** 
.32** \ 
.15 

.46** 



.23** 
«.25** 
.21** ' 
.22** 



.23** 
.27** ' 
.14*, 

.34** 

.17** 
.14* 

• ♦ 

f * 

.17** . 

% 

.16** 
.21** 



* significant at .05 level 
** significant at .01 level 
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Table V-6 



Correlations Between >Apfitude Tests and 'Job Knowledge Test 
for Cartographic Technicians by Ethi^c Group (TOPOCOM Sample) 



^est 

• m 

Coordination 

Hidden Figures 

Vocabulary * ^ 

Object-Number 

Card .Rotations 
* 

CfS Arithmetic 
(Map Planning 

r 

Surface 
Development 

• Maze^Ti^cing 

» A 
\ » 

Following Oral 
Directions 

Identical " 
Pictures 

^Extended Range 
Vocabulary . 

Necessary 

Arithmetic 

Operations 



Black 
Nr^lOl 

-.06 

< 

.46** 
.28** 
.34** 
.62** 
. ..35** 

r . 58** 

<36** 

.48** 

»31** 
• 50** 

.66** 



Mex-ican- , 
American 

N=99 ' 

.18 
.45** 
.22* * 
.10' 
.34** 
.44**"" 
;..42** 

.55** 

.A6*--* 

■ .50** 
. .40** 
.23* 

.44** 



Caucasian 
N=24l' ■ 



,21**-^ ■ 
.40** ■ 
.39** 
'17** 
.33**- 
.4'3** 
.36** 

.54** 

^37** . 

i54** 

.29** 

.39** 

. 54** 



* significant at .05.'level 
** significant at .01 level 
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^ . . - Table V-7 ' • / 

>• - • ■ . . • -■ \: ■ •■ ■ 

! ^ Correlations. Between Aptitude Tests and Work Sample Composite , 
for Cartographic Technicians by Ethnic Group (TOPOCOM Sample^ \ 

* . * 
^ * rMexican- 

' ' Black. - .American • , Caucasian 

Te'st • N=99 . • . N=97 ' - N=236 



Coordinatipn , '.IS* \ ■ .25* / ■ _ .05 

flidden Figures- • ... ' i35**- .A3**/ .2"9** 

.Vocabulary ' ': • . 22* ' -. 02 , ° ^" .22** 

o'bject-Number ' ■ .05 .03; .11 • 

/Card Rotations *' - ^' .26** *V •^S** ^34** 

CS Arithmetic , * .34**\ •11 

Map Planning " • .2?** - '.39** -eA** 



Surface , ^ 3^^^ . 38*^ ' . .41** 

Development . * . 

Maze Tracing ^ / .30** .45** , .34** 

Following Oral" ' ' \* " ! . 20 • • .41** 

Directions ' - . * ' 

Identical • . * . ^ .31** .37** , >26** 

Pictures 

- ' I. ' . 

Extended Range i8 V .05 . .23** 

Vocabulary . . .• . » • . - ' 

Necessary . , > 

Arithmetic ^ ' - , .2£r** . .39** ' ..35** 

Operations ^ - * , • . * ^ ' 



* * significant at .05 level 

** significant at .01 lev^el ^ 
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> Table V-8 

Correlations BetWeen Aptitude; Tests and Work Sample Overall Score ^ 
for Inventory Management Specialists by I^Jfehitic Group 



Test 



Blick 
N=99 



Mexican- ' 
American 

N=58 



Caucasian 



-Number Comparison 

Hidden Figures 

Vocabulary , ^ 

;6BJect-Number • 

tetter Sets ' ' 

Nonsense Syllogisms 
% 

Subtraction & 
Multiplication 

Extended Range 
•Vocabulary 

Necessary 

Arithmetic 

Operations 

Following Oral 
Directions 

Inference 

FSEE (VA + QR) 



..17 
.21* • 
.32** 
- .04 
.28** 
.29*"* 

.08 

128**- 
.33** 

.36**' 

.39** 
.37** 



.36** 

.29* 

.41** 

. 20 ; ■ 

.'49**' 

.38** 

• .37** 
.58** 

• .60** 

.41** 

.56** 
.60** 



.34** 
.30**. 

.06 

.29-** 

•13 

.13 

.32** 
.35**. 

.42** 

.34** 
.40** 



* significant at .05 level 
** significant at .01 level 
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^Chapter VI 

Regression Analysis of Belst Predictors of Job Performance hfeasures 

for Different Ethnic Groups 
Resdits reported in the previous chapter show that aptitude tests 
tiaye substantially equal validity for the three ethnic groups, regard- 
less of the type of criterion used. However, the question of - test 
fairness demands that other aspects of the relationships between test 
scores and measures of job performance be considered as well. 
Specifically, the regression lines for different ethnic groups must 
Be compared. These lines show the linear relationships between test 
scores :and crit^ measures, and if the lines for two ethnic groups 
differ significantly, this indicates that the same iest score is pre- 
dicting different , criterion scores for the two groups. 
Signi^ficance Tests for Regression Lines ^ - ^ 

-In comparing two regression lines,* three different aspects of the 
relationship are important: * \ 

(1) How much scatter is there around the regression lines? 
(bo the standard errors of estimate differ?) 

^(2) Do the slopes of the regression* lines differ? (Is, there a 
closer, relationship fbr one group, than for another? This is almost, 
but not quite, the same question considered in the previous chapter, 
since the standard deviations of both. variables, as well as the 
correlation coefficients affect the slope.) 

•(3) Do the intercepts differ? (Is one regression line generally 
'higher than the other?) < . - ' 

Table VI-1 summarizeis the results of this kind of comparison for 

-157- 
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\ the three studie^s in this project • The Gulliksen-Wilks. tests for 

significance of differences between pairs of regression lines involve 
' three successive tests: (1) for differences in 'scatter or dispersion 

he' 

around the regression lines; (2) if test (1) is not^significant, then 
for; differences in slope of the regression lines; arid (3) if test (2) 
is ^Isp not significant, for differences in intercept , or g«fieral level 
of the regression line, ^ ' 

Table Vl-i shows that when the criterion is Supervisors' Overall 
^ ;Rating, there are very few differences in regression lines. Out of 72 
; pairs of regression lines, comparing the line for Caucasians, with the 
jSiie for each minority group separately^, four are significantly 
different .at * the .05 level, or better. Three of. these differences are 
in slope and one/in dispersion. 

. When the' criterion is the Job Knowledge Test, on .the other hand, I 
out of* 35 comparisons, 31 pairs are significantly different. All blit 
two of these are differences in intercept.. 

With a Work sample as the criterion, 38 pairs 'out of 50 compari- 
sons show significant differences. Of these, 22 differences .are in 
dispersi'on,- three in slope, and 13 in intercept. 

In the case of the Inventory Manager samples, a further step was 
taken. For' the first two occupations, investigation of response 
patterns to the task list items showed that the different ethnic groups 
"were doing essentially the same job. In the case of the Inventory 
Managers, while comparison of. Blacks and Caucasians showed few signifi- 
cant differences in task list responses, comparison of Mexican-Americans 
and Caucasians showed a substantial number' of significant differences. 

130 



However, when the Mexican-*Aicerican Inventory Managers, who were all 
located at San Antonio, were compared with Caucasians at San Antonio , 
there were very few task list items showing significant differences 
in response patterns,, ^Excluding the San Antonio Inventory Managers, 
comparison of Blacks and Caucasians at the other foyr installations 
showed fewer task list' items with significant differences than would 
have been expected by chance, (These findings will be described more 
fully^ in Chapter X •) Thus, it seemed appropriate to compare regres- 
sion lines for Caucasian arid Mexican-American Inventory Managers at 
'San Antonio separately from the Caucasian and Black Inventory 
Managers at the other, four ""installations. 

Table VI-2 shows these comparisons^ With Supervisors* Overl^jp' 
Rating as the criterion, only two^out of 12 pairs of regression lines 
for Blacks and Caucasians show significant ^differences at the .05 
ievei or better, both of these in the intercept. There are no signif 
ipant differences .in the regression lines for Mexican-Americans and 
Caucasians. 

With the Work' Sample Overall score as the criterion, all 12 
pairs of regression lines for Blacks and Caucasians*, ^how significant 
differences, all in dispersion around the regression lines. .This 
reflects the fact that the standard deviation fpr the Work Sample 
Overall score is' .3.4 for Black Inventory Managers and 4.4 for 
Caucasian Inventory Managers. / 
Comparison of Regression Lines by Criterion Measure 

Too 'many regression lines are involved to cOTsider them all 
individually in this chapter. Figures are present 
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regresslon lines for those aptitude tests with the highest validity 
for each type of" criterion measure for each ethnic group for the 
three occupations studied* Some charts will be included where 
Gulliksen-WilEs^ tests referred to in the previous >section found no 
significant differences between^he pairs of regression lines, as 



well as Some- where there were significant differences, (It will bev 
apparent that visual inspection of the Vegression lines is not*" 
sufficient to determine whi(ih pairs of regression lines are signif-^ 
ica fitly different.) , . ' ' 

Supervisors* Oveirall Rating as Criterion 

Figure VIr-1 shows the regression lines for the Necessary \^ 
Arithmetic pperatlons *Test against Supervisors* Ovelrall Rating for 

' Black and Caucasian Medical Technicians. This test had the highest 
correlation with Supervisors* Overall Rating for Caucasian Medical 
Techn^ians. Since there were no significant differences between the 
Black and Caucasian regression lines, it can be concluded that a given 

-test score 'will predict essentially the sarae^ criterion score for both 
groups. 

Figure VI-2 shows similatr' regression lines for the Subtraction 
and Multiplication Test, which had the highest validity for Black 
Medical Technicians. Here, the slopes of the two regression, lines 
show a difference significant at the .05 level. If the Caucasian 
regression line were to be used in selecting BlacTcs, criterion scores 
for those with high test scores would be under es t ima ted , while 
criterion scores would be overestimated for thosc'^with low test secures. 

figure Vr-:3" shows the regression lines for the Map Planning Test 

5. iarg 



- - - ^ . ^ V 

agafnst Supervisors' Overall Rating for Black, Mexican-American, and ^ 

Caucasiarv Cartographic Technicians. Tliis test had the hlghest-valid- 

• **« * ' - • ' f . ' " ' ' ' 

ity ipr the Caucasian sample. Here, neither the Black nor the * , • 
Mexican- American regression line is significantly different from the 
Caucasian regression line, so' that it would appear that various test , 
scores have the same meaning in relation to the, rating criterion for 

- , ■ ■ ' ' . : 

ali three groups.- ' • , ■ f 

The same statement can be made- about Figure >VI-4, which shows 
* the regifessioh dines 'ifor the Civil Service Aritlimetic. Testi the most/ 

vali'd test, for Blacks, and .about Firfeure which shows the regres- 

^'sioiir lines for the Hidden Figures Test, the most« valid test for 

Mexican-Americans. ' . . " . 

Figure VI-6' shows the regression lines for the Subtraction and' ^ 
' " . , ' 

Muitiplication Test against Supervisors/ Overall Riifting for Black, 
. Mexican- American, and Caucasian Inventpry Managers. This test^^.h^^ 
•the highest validity for Blacks and Caucasians. ^Again, there is no 
significant differeiiice between the pairs of regression lines, so that 
the test scores appear%o> have the same meaning for the various groups. 

Figure VI-7 shows the regression lines for the Nonsense Syllo- ^ 
gisins Test, which had the highest validity for Mexican-American 
Inventory Mjanagement Specialists. Again, there are no significant 
differences between the Caucasian regression line and eitheV of the 
minority group regression lines. 

Job Knowledge Test as Criterion 

The Necessary Arithmetic Operations Test had the highest cor- 
relations with Job Knowledge Test scores for both Black and Caucasian 



Medical Technicians. These regression lines are shown in Figure VI-8, 

In this ins^tance, the slopes of the two line^ are significantly, differ- 
• • • *^ ' 

ent at' the .01 level. Thus, if the Caucasian regression line were 
%> ■ ' ■ 

used to predict Job Knowledge Test scores for Blacks, criterion scores 
for those Blacks with high test scores would be slightly underestimated, 
while criterion scores for those with low test scores would be over- 
estimated to a somewhat greater extent. 

For Caucasian Cartographic Technicians, three tests had equally 
high validities against Job Knowledge Test scores. These three tests 

— M 

were the Surface Development Test, which also had the highest validity 
for Mexican-Americans, Following Oral Directions, • and Necessary Arith- 
metic Operations, which also had the higliest validity for Blacks. 

Figure VI-9 shows the regression lines for the Surface Development 

Test for Black, Mexican-American, and Caucasian Cartographic Techni- 

* - .J " * • 

cians (TOPOCOM) . In this instance, there are no significant 

V . ' . . - 

differences between- the lines for Blacks and Caucjisians. There is a' 

significant difference in the intercept between Mexican-Americans and 

Caucasians, that is, the general level of the- regression line above 

the base line. Thus, if the Caucasian regression line were used to 
*/ • 

predict criterion scores for Mexican-Americans, their performance 
would betomewhat overestimated. 

Eig^e VI-10 shows the regression lines for the Necessary Arith- 
metic Ope^^ions Test. Here again, there are no significant differences 
between the Black and Caucasian regression lines; but between the 
Mexican-American and Caucasian regression lines there is a difference 
in intercepts significant at the .01 level. Again, use of the 
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Caiica^sian regression line for Mexican- Americans would overestimate their 
ScritTerion performance. • ' * 

• A .similar situation obtains for the Following Oral Directions Test, 
shown^ in Figure VI-11. Here again, the Caucasian regression line would 
overestimate the flexican-American criterion score. 
Work Samples as^Criteria 

The ^Hidden Figures Test had. the. highest correlation with the Work 
Sample Composite score for Black Cartographic Technicians; .the Ka^ze 
^Tracing Speed Test had the highest correlation for Mexican-i^mericans; 
.and two tests,. Surface Development and Following Oral Directions, had 

equally high correlations for Caucasians, 

- . ^ 
Figure VI-12 shows the regression lines for the Hidden Figures Test 

'for Black, Mexic^in-American, .and Caucasian Cartographic Technicians, 
There- is a difference in slopes between the Mexican-American and Cauca- 
sian regression lines, significant at the .05 level; and a difference 
in Intercepts between the Black and Caucasian regression lines,' signifi- 
cant at the .01 level. As can be seen, the Caucasian, regression dine 
, lies'above the regression lines for both minority groups. - / 
Figure VI-13 shows the regression* lines for the Maze Tracing-'Speed . 
Test. Again, the line for Caucasians lies above .the lines for the two 

' ' ■ . . / 

minority groups. There are no significant differences between the lines 

/ ' - ' 

for Blacks and Caucasians, but there is a difference in intercepts, 

• / 

significant at the i05 level, between the Mexican-American, and Cauca- 
sian lines. 

/ ■ - I 

Figure VI-14 shows the regression lines for the Surface Development 

Test. Again, the line for Caucasians lies above the lines for the two 
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minority groups. There arfe. no significant differences between the Black 
and Caucasian regression lines, but ths're is a difference in disp^ersion, 
■signff icarit at the .01 level, between the lines for the Mexican-American 
and Caucasian samples, - . %, . 

"Figure VI- 15* shows the regression lines for the Following Oral 
Directions Test. There, is a difference in slope between the Caucasian 

aifd^Slack regression lines, significant at the .05 level. Between the 

/ ' ^' * - " , ' 

Mexican-American and Caucasian regression lines, there is a ,diff er^ice 

in dispersiori*j^ significant at the .01 level. Although use of the Cauca- 

, • • • 

sian regression equation would overestimate the criterion ^scores for 

those minority incumbents with high test^ scores, the criterion scores of 
low-scoring«minorities would* be somewhat underestimated. 

" The most valid ^est for Black Inventory Managers against Work 
Sample Overall Performance was the Inference Test; for Mexican-Americans 
the Necessary Arithmetic Operations Test and the Verbal and Quantitative 
Reasoning sections of the Fedferal Service Entrance Examination had 
equally high validities; and for Caucasians the Following Oral direct- 
ions Test was most valid. ^ 

Figure VI-16 shows the regression lines for the Inference Test. 
There are differences in dispersion between the Caucasian regression 
line and both the Mexican-American (significant at the •05. level) and 
the Black (significant at the .01 level) regression lines. Of perhaps 
more interest is the fact that the Mexican-American regtession line 
lies almost entirely above the Caucasian regression line. In this 
instance, fchen, the use of the Caucasian regression line for Mexican- 
Americans would seriously underpredict their criterion scores. 



A somewhat similar situation is seen in' Figure VI-17, which sho*s 

the regression lines for the Federal Service Entrance Examination (V + 

Q),* .Again, the differences in dispersion between ^he Caucasian regres- 

» * * , • 

sion line and the two minority regression lines are Significant, at the 
.01 ieV^l between the Blacks and Caucasians and at the, .05 level betWeen 
the Mexican-Americans and Caucasians. And again^, the Mexican-American 
' regression line lies almost entirely* above .the Caucasian regression' 
line. - . . U ' < . t 

Figure VI- 18 shows the regression lines for the Necessary Afith-. 
metic Operations Test. - Again, the regression line for the Mexican- 
American group lies above that for the Caucasian'-group, which, in turn, 
^is above that for the Blacky group. There are significant differences 
^in , dispersion, at the .05 level between the Caucasian and' Mexican-'; 
Americans and at the .01 level between Caucasian^ and Blacks. 

Figure yi-19 shows the regression lines f or ^the^^llowingcOral ' 
Directions Test. The relative placement of the regression linels xs 
similar to the configuration in the previous figure. In this instance, 
' .there are, no significant differences between the regression lines ''for 
Caucasians and Mexican- Americans, but between the Blacks and Caucasians 
there is, again, a difference significant at the .01 level* 

It-appears, then, from this analysis, that. In predicting Work 
Sample Overall Performance, use of a Caucasian prediction equation 
would discriminate against one of the minority groups. 

Additional Analyses for Inventory Managers by Installation 

AS indicated earlier, separate regression lines and Gulliksen- 
Wilks comparisons were made for four subgroups: Mexican-American and 
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Caucasian Inventory Management Specialists at San Antonio, and Black 
and Caucasian Inventory Management Specialists at. the Philadjelphia, 
Dayton, and Detroit installations. 

With Supervisors* Overall Rating 'as the' criterion, the Nonsense 
' Syllogisms Test was most predictive for Mexican-Americans, the 
Necessary Arithmetic Operations Test was^ most predictive for San . 
' . Anjtoriio' Caucasians, and the Subtraction and Multiplication Test was 
, ^ most, valid, for both Blacks and Caucasians at the other installations. 
' Figure Vl-20 shows the regression lines for the four groups on 
, the Necessary Arithmetic Operations Te7st. There are no significant 
• ^ff efences between the Mexican-American and the San Antonio Cauca- 
^sian regression lines. The regression lines for Blacks and Caucasians 
at other installations have intercepts ^si^ificantly different at the 
.05 level. Jlere, the Caucasian regression- line would underestimate 
the. criterion scores for Blacks.' 

* * . Figure VL-21 shows the regression lines for the Nonsense Syllo- 

gisms Test. . Here, there are no significant differences between the 

degression lines for Mexican-Americans and San Antonio Caucasians or 

I 

tjjte lines for Blacks and Caucasians at other installations. 

J^igure VI-22 shows the regression lines for the Subtraction and 

• Multiplication Test.' The Mexicans-American and San Antonio Caucasian 
lilies are not significantly different. The lines for Blacks and 
Caucasians at other .installations have intercepts different at the 
.05 level. Again, th*e Caucasian regression line would underestimate 
the Supervisors* Overall Ratings received by the Bl^ck Inventory ^ 
Management Specialists. • ' - ^» 
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With Work Sample Overall Performance as the criterion the 
•Necessary Arithmetic Operations Test and the Federal Service Entrance 
Examination had equally high validity for the Mexican-American 
Inventory Managers, ihe Necessary Arithmetic Operations Test and the 
Vocabulary Test had equally high validity for the San Antonio Cauca- 
"sians, the Inference Test had highes^t validity for Blacks from other 
installations, and the Following Oral Directions Test had the highest 
validity for Caucasians from other installations. 

• Figure VI-23 shows the regression lines for the Inference Test. 
There are no significant differences between the Mexican-American and 
San Antonio Caucasian regression lines. The regression lines for 
Blacks and Caucasians, from other installations had dispersions . 
significantly different at the .01* level. In this instance, the 
Caucasian regression line would overpredict the Black criterion perfor- 
mance « 

Figure VI-24 shows the regression lines for the Necessary Arith- 
metic Operations Test. 'Again, there are no significant differences 
between the Mexican-American and Saii Antonio Caucasian regressi9n 
lines, and a difference in dispersion, significant at the .01 level, 
betweei> the regression lines for Blacks and Caucasians at other 
installations. And again, the u^of the Caucasian regression line 
would overestimate the criterion performance of Black Inventory 
Managers. 

- ' Figure VLr25 shows the regression lines for the Following Oral 
Directions Test. Again, there are no significant diffe^rences in the 

lines for the Mexican-American and San Antonio Caucasians. There is 

m 
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a difference in dispersion between the regression lines for the Blacks 
and Caucasians at other installations, significant at the .05 level. 
The Caucasian regression line would overestimate the criterion perfor- 
mance of Black Inventory Managers ♦ 

Figure VI-26 sHows the regression lines for the Federal Service 
Entrance Examination, and Figure <^^-27 the regression lines ^Eor the 
Vocabulary Test. For both^ of these, there are no significant differ- 
ences between the Mexican-American and the San Antonio Caucasian 
regression lines. The regression lines for Blacks and Caucasians from 
other installations have dispersions significantly different at the .01 
level. Again ,^ the Caucasian regression line would^ overpredict the 
Black criterion performance. ' -4 . 

in most of the regression line comparisons that have been made, 
use of Caucasian regression lines to predict criterion measures for 
minority groups would result in bias in favor of the'^minorities; that 
is, higher criterion scores would be predicted by use of Caucasian 
regression lines than those predicted from their own regression lines. 
Hov/ever, when regression line comparisons are made between Mexican- 
American Inventory Managers and Caucasian Inventory Managers, with 
Work Sample Overall Performance as the- criterion, the bias is in the 
other direction. This bias disappears when the comparison is limited 
to the regression lines for Mexican-Americans and Caucasians from San 
Antonio. ' 

However,' the comparisons between Black and Caucasian Inventory 
Managers at the Philadelphia, Dayton, and Detroit installations, with 
Supervisors' Overall Ratljag^as the criterion, now shows a similar bias 
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against the Blacks. .This difference reached the significance level for 
only two of "the 12 regression lines. 
Contingency Tables for Selected Tests , 

there is another aspeot-^o4«.the^probXem of differential prediction 
which should be considered. A great many, if not most, valiSlty studies 
are dgne with supervisors* ratings as the sole criterion, and with a 
largely, if not exclusively, Caucasian group as the validation sample. 
What happens, then, when attest, established as valid in that situation, 
fs: used to make predictions for other ethnic groups or is compared with 
other types "of criteria? 

' Tables »VI-3, VI-4, VI-5, and VI-6.are responsive to these questions, 
-In each of these tables, the score range for the test in question^. has 
been divided into four intervals. Succeeding columns show the mean 



cr^-teriOTi score for those persons in each of the four intervals. Table 
\VI-3 shows the data for the- Medical Technician sample. The Necessary 

\ -. 

Arithmetic Operations Test, which best predicts the Supervisors* Overall 
Rating for the Caucasian sample, also produced valid discrimination on 
that criterion for the Black sample'. With Job Knowledge Test scores as 
the criterion, there is again valid discrimination for both samples. 

Table VI-4 shows similar data for Cartographic Technicians from 
TOPOCOM installations. Here, the Map Planning Test produces valid ^ 
discrimination on Supervisors* Overall Rating for Black and Mexican- 
American technicians, as well as for' Caucasian technicians, although- 
those in the fourth test score interval have slightly higher supervisors* 
ratings than those in the third. - 

Table Vl-5 shows the relationship between the Map^ Planning Test 
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scores >and Supervisors* Overall Rating .for Cartographic Technicians 
in the Coast ^nd Geodetic Survey ^sample. On the. whole, valid discrimi- 

/nation was produced for both the Caucasian jand Black technicians. There 
ls> however, one very substantial reversal {(for Black -technicians in the 
highest scofe interval), but there were only two individuals in this 
catiBgory^. • . | . * /> 

Table VI-6 shows comparable inform^ tioji for Inventoj^^ Management ^ 
Specialists. Here, the "most valid test for Caucasian Inventory Managers 

-against the Supervisors' Overall Rating was the Subtraction and Multi:- 

plication Test. Generally valid discriminations are made for all three 

ethnic groups and for both criteria. There; are, however, five reversals. 

One" involves the Supervisors' Overall Rating and is relatively small. 

The other four are for the Work Sample, two for Blacks,--and one each for 

Mexican-Americans and Caucasians. *Three of! these reversals are fairly 

' ' ' . ] 
large. *" . , ! # 

. Despite these discrepancies, however, the general picture is of 

valid discrimination, regardless of ethnic group or criterion used. 

Summary ' 

Regression lines between aptitude tests and three different kinds 



of criteria ^ave been compared for Caticasians and the two minority 

\ ' I . I 

groups\ separately. With Supervisors' Overall Rating as the criterion, 

\ • : ' , 

very feW of the regression lines were significantly different; that is, 

1 » 
essentially the same predictions were made for all groups. 

■ ' ■ ' ' \ 

v4 There were more instances of significant differences in regression 
lines when the Job Knowledge Tests ot the Work Samples were used as 
criteria. The "bias" in almost all instances was in favor of the 
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minority groups rather than against them. 

The general picture that emerges is one of tests either making 
unbiased predictions or of showing bias in favor of the minority 
group's. 
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Table VI-3 

Criterion Score Means for Medical Technicians 
at Different Score Levels on the 
Necessary Arithmetic Operations Test 



Test Scores 



18 + 



Mean Supervisors' 
Overall Rating 



Black 

7.2 

N=5 



Caucasian 

6.3 

N=69 



Mean Job Knowledge 
Test Scores 



Black 

39.4 

N=5 



Caucasian 

40.2 

N=69 



14 - 17.9 



5.9 

N=28 



6.0 

N=81 



38.0 
N=28 



37.3 

N=81 



10 - 13.9 



5.9 

N=56 



6.3 

N=77 



32.6 

N=56 



34.4 

N=77 



- 9.9 



5.7 

N=60 



5.5 ' 

N=55' 



26.9 

N=60 



31.3 

N=55 
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Table VI-5 

j 

Criterion Score Means for 

Cartographic Technicians (Coast & Geodetic Survey) 

on the'Hap Planning Test — j * 

Mean Supervisors* 
> Overall jRating 



Caucasian 



Test Scores Black / C 



24 + ' • 3.8 / 7*0 

N=2 X / N=ff* 



17 - 23.9 7.0 ./ 6.8 

N=10 /' N=18 



10 - 16.9 — 5.4 / ' 6.1 

N=18 



- 9.9 4.2 5.0 

N=6 
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Table VI-6 

Criterion Score Means for Inventory Management Specialists 
at- Different Score Levels on the 
Subtraction and Multiplication Test 



Test 
Scores 



Mean Supervisors' 
Overall Rating 

Mexican- 



Mean Work Sample 
Overall Rating 

Mexican- 



Black American Caucasian Black American Caucasian 



90 + 



7.2 

N=14 



7.1 

N=14 



6.9 

N=54 



6.2 

N=14 



9.1 

N=10 



9.4 

N=43 



70 - 89 



6.3 

N-28 



6.4 

N=22 



6.4 

N=65 



6.7 

N=24. 



11.3 
N=18 



8.4 

N=56 



50 - 69 



6.4 

lN=48 



6.3 

N=23 



5.8 

N=48 



7.1 

N=44 



8.4 

N=18 



8.6 

N=40 



- 49 



4.8 

N=22 



5.6 
,N=13 



5.3 

N=20 



5.7 

N=18 



5.9 

N=ll 



8.3 

N=18 
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^ . Chapter VII 

Multiple Cb/rrelations and Cross-Ethnic Ctoss-Validation 
I Coefficients for Prediction of Criterion Measures 

In Chapter V, the relationships of the separate aptitude tests in 
the predictor batteries with the criterion measures were compared by 
ethnic group for each occupation studied. Aptitude tests were shown 
to have substantially equal validity for the different* ethnic groups. 
In Chapter VI, the regression lines of best predictors for each ethnic 
group in the three occupations were compared with regression lines for 
these predictors for the other ethnic groups. 

. • ' / 

In this chapter, the results of stepwise multiple regression analy- 
ses are described, in which a best set of predictors was selected by 
ethnic group for each criterion measure. The prediction equations thus 
derived for each ethnic group for each criterion measure were then^p:^ 
plied to data for the other ethnic samples to provide cross-ethnic 
cross-validation coefficients in further examination of differential 
validity. 

In each occupation stucjied, the set of predictors was restricted to 
no more than three for each criterion measure. Predictors with negative 
regression weights were not used, so that for some criterion measures, 
only one or two predictors were included, selected prior to the entry of 
a negative predictor. In no case was a negatively weighted predictor 
selected first. Holding the number of predictors selected to three or 
fewer reduces the possibility of chance or spurious relationships enter- 
ing into an inflated multiple, and increases the likelihood of relation- 
ships holding up in cross-validaCion. Correction for "shrinkage" in 
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the multiples due to sampling errors is also minimized when the number 
of variables included is small."'" 

The results' for each of the occupations studied will be discussed 
separately. The research questions to be explored are: 1) How do pre- 
diction equations differ (composition, weights, and level of predictipn) 
for different ethnic groups? 2) What 1$ the effect of " using prediction 
equations derived from one ethnic group sample on data for another ethnic 
group (cross-ethnic cross-validation/)? 3) What is the effect of cross- 
ethnic prediction of criterion scores'where test scores of individuals 
fall one standard deviation below the. mean, at the mean, or one standard 

deviation above the mean of/their group? 4) Is the use of differential 

/ . ,^ ^ 

regression equations for ethnic groups warranted by evidence of bias in 

> 

prediction? 

Medical Technicians .--^ 

fable VII-1 shows the multiple correlation coefficients (R) and 
standard, regression weights derived" for sets of predictors selected for 
the criterion measures of supervisors* ratings of Learning Ability, 
Technical Knowledge, and Overall ef f ectit^eness, and the Job Knowledge 
Test, for Medical Technicians. Although multiple correlation coeffi- 
cients were obtained for all rating scales, only Learning Ability and 
Technical Knowledge ratings are being reported here in addition to 
Overall rating, as representing supervisors* judgments of less sub- 
jective aspects of job performance. These two scales had as high 
relationships with predictors as the Overall rating (avid in some cases - 
higher) for all occupations studied. ■ 

"'' Correction for shrinkage" in the multiples for the smallest sample in 
the present study (N = 57) amounts to no more than .02. 
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jlt can be seen In Table VII-1 that multiples and regression weights 
for^the Black sample are higher in every case than for the Caucasian 
sample. Higher multiples were obtained for the two rating scales than 
for the Overall rating- for the Black sample and on the Learning Ability 
rating for the Caucasian sample. Tnese differences were to be expected 
from the differential validities reported in Tallies V-1 and V-5 in 
Chapter V, where^vthe correlations of aptitude test scores with criterion 

^measures for the Black sample were higher in a number of instances than 
for the Caucasian sample, "^especially true for the Job Knowledge Test. 

, There is some consistency in the composition of the sets of predictors 
across criterion measures and ethnic groups. Necessary Arithmetic Oper- 
ations (a general reasoning test) and ^he Subtraction & Multiplication 
Test (number facility) appear selectively as predictors of supervisors' 

ratings and Job Knowledge Tes; both ^ethnic groups. 

/ 

Table VII--2 is a comparison ^ multiple correlation coefficients 

/ 

and cross-ethnic cross-validation coefficients for the same criterion 
measures shown in Table VII-1, with weights derived for the Black sample 
used to predict" multiples for the Caucasian sample an^^vice versa. The 
multiples obtained for the Black sample using Black weights and Cauca- 
sian sample using Caucasian weights, ^shown in Table VII-1, are included 
again in Table VII-2 for comparison.^' The multiples for the Black sample, 
using Black weights, are substantialiy higher than for the Caucasian 
sample using Black weights, but the reverse^ is true when Caucasian weight 
are used to predict multiples for the Black sample. The multiples for 
the Black sample thus obtained are higher in every instance than Cauca- 
sian sample multiples based on Caucasian weights, although no^^ greatly 

/ 
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SO, except for the Job Knowledge Test. There is a fair amount of shrink*- 
age in the multiples for the Technical Knowledge rating (from .38 to .21) 
and Overall rating (from .29 to .17), using cross-ethnic weights, but the 
multiples for the Learning Ability rating (.42 to .40) and the Job Knowl- 
edge Test (.53 to .52) hold up whether Black or Caucasian weights are 
used . 

Table VII-S shows scores for the same criterion measures predicted 
from multiple regression equations for ethnic group samples where test 
scores in the equations are one standard deviation below the mean, at the 
mean, or one standard deviation above the mean. Results are shown where 
Black weights were used alternatively on scores from the Black and Cau- ^ ^ 
caslan samples and Where Caucasian weights were used alternatively on 
scores for fhe Black and Caucasian samples. For all four of the criter- (^^y^ 
ion measures shown, it can be seen that Blacks with high scores (one 
standard deviation above the mean) tend to have higher predicted scores 
'When Biack weights are used, but those with low scores (gne standard 
deviation below the mean) tend to have slightly higher predicted scores 
when Caucasian weights are used (except on the Learning Ability rating, 
which is slightly lower). It can also be seen that in nearly every 
instance the 'predicteci scores of Caucasians scoring below, at, and 
above the mean on aptitude tests are higher when Black weights are 
used in the prediction equation. These results ^reflect the earlier 
finding of higher validities (and resulting steeper regression slopes) 
iu the regression equations based on the Black sample, but higher mean 
scores (and thus a larger intercept constant), in the regression equa- 
tions based on tke Caucasian sample., 
\ 
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In summary, for the Medical Technicians, the multiple correlation 
coefficients obtained for the 'Black -sample were consistently higher 
than/for • the Caucasian 'sample and, in general, the same aptitude, and 

^ ' • ■ . . . ■ , ; ■ 

ability factors were represefnted and more heavily weighted for the 
Black sampl^ in the regress j.on equations • In the cross^ethnic cross- 
validation, the weights ^derived from the- Caucasian sample appeared to 
be equally valid for the Black sample on the Learning Ability rating 
-$ind the Job Knowledge Test, although there was some atttition in the 
o.ther ratings. HciwBver, use of the weights derived from the Black 
sample on the Caucasian sample resulted in attrition in all 'multiples. 
Where t^t scores below the mean, at the mean, and 4bove the mean were 
^ used to predict criterion sdoires by ^alternative use of ethnic regression 
weights, use of Black regression equations tenjjled to favor Caucasians 
with -scores at and abcVe^ theVmean, but not those with low scores* Con- 
v^rsely, the use of^«Caucafian regression equations would benefit Blacks 
wi(;h low test scorers »but not those with high test scores. 
Cartographic Technicians (TOPOGOM;^ . - ' 

' Table VII~4 shows multiple corr^ation coe'f f ici^nts and standard 
regression weights for the sets of predictors selected by stepwise 
multiple regression for' supervisors' ratings of Learning Ability, . 
Technical Knowledge, art^^^erall effectiveness. Job Knowledges Test, 
and Work Sample Composite;^^^thnic group. It can be seen that 
while individual tests app^/r in the sets of predictors for more than 
one ethnic group, in general the composition of the sets is different 
for the different ethnic groups for each criterion measure. Cae 
exception is noted, where the £-ame set^ of pred-ictor^, appears for the 



\ 
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Caucasian sample for both* the/job Knowledge Test and the Work Sample 
Composite, The multiples obtained are fairly consistent in size across 

■ 

ethnic groups, with those' £or^ the Black sample as high or higher than 

"^fpr the Mexican7American and Caucasian sample, except for the Work 

Sample Composite, where the Me?ci-can-American multiple is higher than 

the other two. It is interesting to note the mix of aptitude and skill 

« 

factors in the sets of predictors for.^this occupation by ethnic group. 
For example, the best predictors for the 31ack sample on the Job Knowl- 
edge Test (for which exceptionally high multiples were obtained for all 
three groups) were* CS Arithmetic, Extended Range Vocabulary, and^ 
Necessary Arithmetic Operations (general reasoning), with CS Arithmetic 
having the largest weight. - For the Mexican-American sample, the set 
includes Surface Development, with the largest weight. Map Planningy' 
and CS Arithmetic. -For the Caucasian sample, again the largest weight 
appears for Surface Development, next highest for Extended Range Vocab- 
ulary, and' tMrd highest for Following Oral Directions. 

•Table VI-I-5^hows^ multiple correlation coefficients and cross- 
ethnic cross-validation coefficients obtained where weights derived 
for eacli of the three etlinic groups were used alternatively to compute 
multiples for each sample. It would be expected that some attrition 
would result when weights derived for one ethnic group are applied to 
samples for other ethnic groups. This occui^s to some degree, but there 
are exceptions. For example, the use of Caucasian weights on ail three 
ethnic samples for the Learning Ability rating produces multiples of 
• 44, .44, and ..43, respectively. Using Black weights alterr^tively, 
multiples of ^.47 (Black sample), .42 (Mexican-American sample), and 
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.40 (Caucasian sample) are. derived. Using Meslicah-American weights, 

* • V 

multiples of .41 -(Black), .48 (Mexican-American), and .40 (Caucasian) 
are obtained. For the Job Knowledge Test, the results are also remark- 
ably consistent across ethnic groups For the Black Seunple, using' 
Black weights, R = .74; using Mexican-American w^eights,, R = .63;. and 
using Caucasian weights^ R = .68. 'For the Mexican-American sample, 
using Mexican-American weights, R = .61; using Black weights, R = .52; 
and using Caucasian weights, R = .59. For tha Caucasian sample, using 
Caucasian weights, R =' .66; using Mexican-American Wi^ights, R = .60; 
and -using ^Black weights, R = .61^--^, . . 

The results are not quite as consistent. for the Technical Knowl- 
edge and .Overall ratings and the Work Sample Composite. Attrition in^ 
the multiples for the Technical Knowle.dge rating for the Black sample 

f^M, .47 to .29, and for the. Mexican- American sample from .30 to .20, 

*"* * , " 

result when Caucasian we!ight;s are used. The Caucasian multiple 
* ' ; . / 

changes from R = ,29 using Caucasian. weigjhts to .28 (using Me:^ican- 

American weights) and .19 (using Black weights). 

For the Overall rating, Caucasian weights applied to data for 

Mexican- American -and Black samples produces multiples "that" arfe as 

high or almost as -high as when respective weights for these groups.^ 

♦ 

are applied to their own data. Conversely, when Black weights are 
applied to data for Mexican-American and Caucasian samples, attri- 
tion is fairly great: Mexican-American (Mexican-American) = .28, 
Black (Mexican-American) = .16; Caucasian (Caucasian) = .35, Black 
(Caucasian) =,.17. The attrition when using Mexican-American weights 
is also fairly great: Black (Black) = .36, Mexican-American (Black). = 
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1 

.24; Caucasian (Caucasian) = .35, Mexican-American (Caucasian) = .20. 

For t;|ie Vork Sample Composite, applying Caucasian weights to data 
fof" Black and Mexican-American samples results in some attrition in the 
multiples obtained when using the regression weights obtained for th^ir 
own samples: Black (Black) = .39, Caucasian (Black) = .30; Mexican- 
American (Mexic5ln-American) = .53, Caucasian (Mexican-American) = .33. 
U^ing Black and Mexican-American weights alternatively on the other two 
ethnic groups results in less attrition. 

Some insight into the fluctuations as the result of cross-ethnic 
cross-validation may be obtained by examining the sets of predictors for 
the Various criterion measures by ethnic group, shown in Table VII-4. 

At the bottom of Table VII-5, the i^ultiples and cross-ethnic multi- 
ples are shown for supervisors' Overall rating for the Coast & Geodetic 
Survey sample, xising weights derived from the Black and Caucasian TOPOCOM 
samples. Multiple regression analysis was not carried out for the Coast 
& Geodetic Survey sample because of the small size of the ethnic group 
samples. Supervisors' ratings were the only criterion measures avail- 
able for thi^ sample. In general, both the Caucasian and the Black 
TOPOCOM weights produced higher multiples for prediction of supervisors' 
Overall^ rating for 'the Coast & jGeodetic Survey sample than for the 
TOPOCOM sample, suggesting thaf the TOPOCOM regression equations are 
.equally valid for this group. 

Table VII-6 shows predict-ed criterion scores using ethnic and 
cross-ethnic regression equations where test scores of the respective 
ethnic samples are one s^tandard deviation below the mean, at the mean, 
and one standard deviation above the mean. 
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f It can* be seen that the Black group with scores below the mean 
tend to have higher predicted criterion score's when Caucasian weights 
are us^d for all criterion measures shown. For example^ on the Job 
Knowledge Test, the Black equation used on Black scores predicts a 
score of 24.7; the Caucasian equation predicts 26.0 This is also 
true at the mean. For ratings, predicted criterion scores a^'e not as 
consistent for high scorers, but differences are small. For the Job 
Knowledge Test and the Work Sample Composite, again Blacks scoring 
high tend to have slightly higher predicted scores when the Caucasian 
regression equations are used. For the Mexican-Americans, the results 
are not as clear-cut. Ratings predicted from low scores tend^to be 
lower using Caucasian weights, but Job Knowledge Test- and Work'' Sample 
Composite predicted scores are slightly higher. This is generally true 
for scores at' the mean, although use of Caucasian weights result in a 
slightly lover criterion score for the Work Sample Composite than the 
Mexican-American weights (17.0 from 17.7). For high-scoring Mexican- 
Americans, prediction of Overall rating is the g^ame using either Mexican- 
American or Caucasian weights; prediction of Job Knov/ledge Test scores 
IS higher using Caucasian weights (38.9 - 41.4). and lower for the Work 
Sample Composite (25.1 - 22.0). 

For the Caucasian sample. Black regression equations predict 
ratings sl;.ghtly lower for low-scoring Caucasians than Caucasian 
equations, *and Mexican-American weights predict slightly higher, ratings. 
For high-scoring Caucasians, both Black and Mexican-American weights -* ^ 
predict higher ratings than those predicted by Caucasian weights. On 
the Job Knowledge Test, Black weights tend to predict Higher scores for 
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Caucasians who score in both the low and high groups, with more of a 
I 

difference for the high scorers. On the Work Sample Composite, Black 
weights produce lower predicted scores for both low and high Caucasian 
scorers than Caucasian weights. 

In summ^iry, for Cartographic. Technicians, while sets of predictors 

differ for the respective ethnic groups, multiple correlation coeffi- 

\ 

cients were fairly consistent in size' across ethnic groups. Those o 

obtained for the Black and Mexican-American samples were as high as, and 

i 

in some cases higher than, those obtained for the' Caucasian sample. 

In general, the use of Caucasian regression equations on data for 
the other' two samples predicted performance. of Blacks and Mexican- 
Americans with less attrition in multiples than might be expfected in 
cross-ethnic cross-validation. In cross-ethnic comparisons of predicted 
criterion scores where test scores were below the mean, at the mean, or 
above the mean, Caucasian regression equations tended to favor Blacks 
by predicting higher criterion scores for both the low-scoring and high- 
scoring Black groups. This was not as consistently so for the Mexican- 
Americans, but the over- and under-predictions were not great. 

It may be concluded that any bias resulting from use of Caucasian* 
regression weights tended to be in favor of the other ethnic groups, and 
that the use of differential prediction equations for the separate ethnic^ 
groups would not, in general^ bt warranted fur Cartographic Technicians. 
Inventory Management Specialists | 

Table VII-7 shows the multiple correlation coefficients and standard 
regression weights for supervisors* ratings of Learning Ability . Technical 
Knowledge, and Overall effectiveness an4 the Work Sample Compos^^te by 
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ethnic group, for Inventory Management Specialists. The level of predic- 

tion is consistent across criterion measures and ethnic groups, with R 

ranging from .34 to .51 for the Black sample, from .38 to .70 for the ^ 

Mexi-can- American sample, and^.from .35 to .51 for the Caucasian sample, 
ft 

It is. interesting that every aptitude test except Object-Number (associative 

[rote] memory) appears as a predictor for one or more criterion measures 

for one or mdre ethnic groups. This Gonfirms, to some extent, observations 

of the research team regarding the complexity of tK-^ Inventory Manager's 

job, in thart a fairly Vide range of aptitude factors appears to contribute 

unique variance to prediction of performance'. (See validity coefficients 

for individual tests in Table V-4 and V-8 in Chapter V.) 

There 'is little' overlapping in sets of predictors across ethnic groups, 

except fof the Subtraction ,& Multiplication Test (nurber facility) which 

appears as a predictor for ratings in all three groups, but in only two sets 

iEre there as many as two predictors in common (i-,earning Ability rating for 

Blacks and Mexican-Americans) . 

. 

Table VII-B shows the multiple correlation coeffici*' :L3 and cross- 
ethnic cross-validation 'coefficients by criterion measure, where multiple 
regression equations computed for each ethnic group v ;:re alternatively 
applied to t^le data for each othe?: ethnic group. Multiples for within- 
group prediction (shov/n in Table VII-7) are again included for comparison. 

As seen for the previous occupations, the multiples resulting from 
use of Caucasian regression equations do not differ to any marked degree 
from those resulting from use of Bilack weights for Black sample data and 
use of Mexican-American weights for Mexican- American sample data. There 

» 

1 

The Object-Number Test w^s the only test not correlated significantly 
"^'with rating or wdrk sample criteria*. See Tobies VrA and V-8, Chapter V. 
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are greater differences (multiples are lower), especially in ratings, when 
Mexican- American weights are applied to Black and Caucasian data, and whrn 
Black weights are applied to Mexican-American data, but differences are 



less When Slack weights are applied to Caucasian data. It is apparent 

that the most consistent resujLts are obtained when Caucasian weights are 

used for all samples* 

Table VII-9 shows criterion scores predicted when regression equations 

for each ethnic group were used altejrnativery on test scores one standard 

deviation below the meai|, at the mean>^ and on6 standard deviation above the 

mean for* the CLthnic samples. 

Caucasian degression equations used- with 'Black sample data tend to 
.* * 

underpredict ratings across the scoring range, and to overprejiict Work 
Sample Composite scores when compared to those computed by Black regression 
equations. Caucasian regression equations used with Mexican-American data 
tend to predict lower, ratings and lower Work Sample Composite scores across 
the scoring range, when compared with criterion scores predicted by Mexican- 
American regression equations. ^ 

Black regression equations tend to predict higher Caucasian ratings 
and lower Work Sample Composite scores across the range than those predic- 
ted by Caucasian equations. Black regression equations tend to predict 
about the same criterion scores^ for low-scoring Mexican-Americans as 
Mexican-American equations, but lower Work Sample Co;^osite scores. 

^ ' \ ^ ' 

Mexican- American equations tend to predict higher criterion scores 
for both Black and Caucasian samples across the range, when compared to 
scores predicted by each group's own respective equatio^ns, with few 
exceptions. 
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In general, the use of Caucasian regression ^equations favors Blacks 
in the upper scoring range in predicting Work^ Sample criterion scored, 
but not in predicting ratings. In the lower jscoring range, differences 
in predictioii-4re trivial. For Mexican-Americans in both the lower and 
upper scoring ranges, lower Work Sample Comp|os^te scores are predicted by 
Caucasian regression weights than by Mexican-American equations. 

In summary, for Inventory Management Specialists, the level of pre- 
diction was consistent for criterion measures across ethnic gtoups. All 

but one aptitude test in the predictor batjtery was represented at least 

I 

once in the sets of predictors, suggesting the ran^e of aptitude factors 

f ♦ * 

related to j<^b performance in this occupation. Cfoss-ethnic cross-validation 
coefficients showed some attrition in mul^tiples, particularly when Mexican- 
American weights were. applied to Black arid Caucasian data, but results were 
more cbnsistent, with less attrition in multiples, when Caucasian weights 
were used for Blacks and Mexican-Araeric^'ns. When cross-ethnic regression - 
eq.uations were ufeed alternatively to predict criterion scores for ethnic 
subjects scoring below^ at, and above the meariV Caucasian weights favored 



Blacks in the upper scoring rangei Differences were trivial in the lower 
scoring range. However, somewhat lower criterion scores were predicted by 

I * ■ 

Caucasian equations for Mexiekn-Americj'ans in the Wper scoring range on 
both ratings and Work Sample Composite* Differences in the lower scoring 
range were again trivial. 

Prediction and Cross-Validation Across Occupations 

The comparisons for the separate occupations do not take into con- 
side'ration the relative level and -accuracy of predictions attained by 
cross-validation. The multiple correlation coef f icient.s and*cross-ethnic 
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crbss-validatiorv coefficients have been plotted from Tables VII-2, VIi-5, 
and VII-8 for the Black - Caucasian samples (Figure VII-l) and for the 
Mexican-:Americ*an - Caucasian samples (Figure VII-2). 

In each figure, the distance between each point and the diagonal line ^ 
represents the loss in prediction when regression weights from a different 
ethnic group (i.e., Caucasian weights for a Black sample, or vice -versa) 
are used,. It can be seen that, in general, very similar multiples are 
obtained and that' there are no striking discrepancies. 
* , • Summary * * ' - 

For all three occupations, level of prediction achieved was fairly 
consistent across ethnic groups and criterion measures, altho.ugh in some 
instances multiples for Black and Mexican-American samples were higher than 
for Caucasian samples. Learning Ability ratings were, in general, predicted 
at a higher level than Overall ratings, and multiples for these ratings were 
consistently as high as for the Job Knowledge Tests and Work Samples. The 
number of factors represented in the sets of tests selected' by regression 
analysis as predictors was fairly narrow for tUe Medical Technicians, with 
only four out of nine aptitude tests represented in^the sets of "^predictors 
for both achnic groups. For the other two occupations, all of the aptitude 
tests except one were represented in at least one set of predictors for at 

least one ethnic group, demonstrating the validity of the researchers' 

1 

observations of unique job performance factors. 

In cross-ethnic cross-validation of findings, where regression equations 
for each ethnic group were alternatively applied to data for each otfier ethnic 
group, less differential attrition iii multiples resulted for all three r/Ccupa- 
tions when Caucasian weights were used for prediction. For the Me<Hcal 
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Technicians, Black , regression weights produced higher multiples for the 
Caucasian sample because t^e same aptitude and ability factors were repre- 
sented in-, afd more heavily weighted for, the Black sample. However, f or • 
all three occupations, Caucasian regression equations appeared to be about 
equally valid predictors for both Blacks and Mexican-Americans • This hold 
up even where test scores are one standard deviation below, at^ or above 
the mean. Use of Caucasian v/eights resulted in about the same or slightly 
higher predicted criterion scores for fiiacks and Mexican-Americans across 
the range, with some exceptions: It may be concluded that •differential 
regression equations for separata ethnic groups would not be warranted by 
the evidence bf these findings* ^ ^ . . • 
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Chapter VIII- ' • 
Biasing Effects of Ethnic Rater-Ratee Interaction 
For each ^of the occupations studied, ethnic 'identification of super- 
visors^ who completed the rating' forms was obtained, as well as for the job 
incuMb'entr' being rated. Thus it was possible to study the data by ethnic 
group of both rater and ratee to see whether there were any biasing effect 
fromvethnic rater-fatee interaction. 
Differences, in Mean ^Ratings 

Table VIII-l shows the mean rating on each rating scale for the dif- 
fereht ethnic rater-ra tee combinations in the Medical Techriicfan sample. 
BlacK. raters assigned higher mean ratings to Black technicians than to 
Caucasian techgicians oh eight oiit of nine scales. ^Caucasian raters 



assigned higher meah^ ratings to Caucasian technicians' on all nine scales, 
Tabie Vni-2 shows the^mean ratings by ethnic rater-ratee coEabiria- 

. tions for Cartographic Technicians at TOPOCOM installations^^. Again, 
Black raters assigned higher mean ratings to Black technicians thah to 
.Caucasian technicians on seven out of eight rating scales. Mexican- 
Aine^lcan raters assigf^ed higher mean ratings td Mexican-American techrii- 
cians- than to Caucasian technicians on. all eight scales. Caucasian 
raters ^assigned higher mean ratings to Caucasian technicians than to 
Hexican-American technicians on four but of eight scales, and.jiigher 
mean ratings to Caucasian technicians^ than\o Black technicians on all 

, eight scales. ^ . ' . 

AVsomewhat different pattern emerges in Table .VIII-3, which shows 

^^t^e mean ratings by ethnic rater-ratee combinations for Cartographic 



Technicians in the Coast and Geodetic^'Survey. Sample. Here, the Black 

- ^ -277- . \' 
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for these two variables by ethnic group rater-ratee combinations. As- 
can be seen* from the table, Black raters rating Black Medicai Techni- 
^cians assigned considerably higher mean ratings on the Job Knowledge 
"^caie than did Black r^aters rating Caucasian technicians. The Job 
Knowledge Test scores for the twq rating combinations varied in the 
opposite direction. Neither \f ,these differences in means, howe^^rj 
reached the level of statistical significance. The correlation between 
ritiiig score and test was .50 for Black raters irating Black technicians, 
and! .09: for Black raters rating Caucasian technicians, dipse coe^- 
^t^hts are^ignificantly different at the .01 level.. It appears that 
ithe. Biack raters were more lenient when rating. Black .technicians than 
when.ratinjg Caucasian technicians, sor that they considered different 
aspec^ of performance in., rating the two groups. 

,fhe .regression lines for predicting the Job Kn6wl^>^,ge ratings from 
the Job Knowledge Test scores for the "four groups of Medical Technicians^ 

* ' ' . - s 

are shown in Figure VII|-1. The regression line for Black technicians 
rated" by Black supervisors lies above the .regression line for Caucasian ^ 
technicians rated by Black supervisors except at the lower end of the 
test score range. Similarly, the regression line for Black technicians 
rated by Caucasian supervisors lies above the regression line^fo'r 
.Caucasian technicians rated by Caucasian supervisors except at the lower 
endi In both instances. Black technicians with moderate to high Job 
Knowledge Test scores received higher ratings on the Job Knowledge scale 
thah^ did Caucasians with equivalent scores. It is particularly noticeable 
that ratings- assigned to Caucasian te'chnicians by Black supervisors on the 
Job Knowledge scale bear little relationship to the 'Job Knowledge Test 
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^Caucasian supervisors ratings-Caucasian technicians rated them 
slightly higher on the average i^lein they rated Black technicians, 
although , the mean differences ^re hot statistically significant. Job 



Knowledge Test: scores vary inj.the same direction y-' but here the (difference 

is significant, again suggest'ing that the Caucasian raters are possibly 

mare lenient toward the Black technicians. There is a higher correlation 

between test scores and ratings for the Black technicians than for the 

> , . - " ' \' ^' 

Caucasian technicians, but the coefficients, are not significantly 

^different. ^ , • \ ' ^ ^ \ \ 

: 'degression lines for predicting Job Knowledge rat/ings from Job 
Knowledge Test scores for Cartographic Technicians at* TOPpCOM' installa- 
tion^ "are shown in Figures VIII-2, VIII-3, and VIII-4. 

Figure VIII-2 shows the regression 'lines for Black -and. Caucasian/ . 
technicians rated by Blkck supervisors. The two lines arre approximately 
parallel, but the regression, line for Black technicians lies above th^ 
r,egression line for Caucasian technicians, indicating that for equivalent 
test scores. Black technicians receive higher ratings than do Caucasian 
technicians. 

« 

Figure VIII-3 shows the regression lines for Mexican-American and 
JCaucasian technicians rate -y Mexican-American supervisors. Except at-' 
the^upi^t 'end of the scale, the regjession line for Mexican-American 



technicians IjLes above the line for Caucasian technicians. Thus, Mexican- 
<American technicians with low test scores receive higher ratings than do 
Caucasian technicians with equivalent scores. ' ' 

Figure VIII-4 shows the regression lines for Black, Mexicari-Americah, 

* 

and Caucasian technicians rated by Caucasian/ supervisors. The regression 



line for Caucasian technician^ ''lies above the line for Black technicians', ■ 
although the two lines are .fairly close together.- The regression line for 
Mexican-American technicians, lies above the other two regression lines. 
Thus, Mexican-American technicians appear to irecdive higher ratings than 

those of the other "two ^ethnic groups with equivalent test scores. 

Relationship of Learning Ability Ratings and Ob jective Aptitude- and Job 
•Performance 'Measures 

■ It may be useful to consider the relationship between ratings on 
another rating .scale for the various fater-ratee combinations and obj6c< 
tive measures of aptitude and'job perfo.rmance. The'Leaming Ability 
r-rating was selected on the logical ground that this is the aspect of job 
performance which aptitude tests usually can best be expected to predict. 
-m^s expectation is confirmed by. the fact that ratings. on this scale have 
higher correlations with the aptitude tests than do the other ratings for 
allithree occupations studied. " ^ - 

fable VIli-7 shews, for the Medical Technicians, the correlation 
between the Learning Ability rating on the one hand and aptitude test- 
scores and Job Knowledge Test scores on the other, by ethnic group rater- 
ratee combinations. 

■ For the Black raters," higher validity coefficients are shown fo^ 
Black technicians than for Caucasian technicians on five of ten measures. 
"However, the level of yalidity coefficients appears to be higher in 
general for the Black technicians. The average correlation coefficient 
(by r to z transformation) is .27 for Black technicians rated by a Black 
supervisor, and .IJ for Caucas'ian technicians rated by a Black supervisor. 
For Caucasian raters, similarly, the validities for Black technicians 
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were higher than for Caucasian technicians on five out of ten tests-. ^The 
Average coefficient for the Black technicians is .25 and For Caucasian 
•technicians .23. 

table VIII-8 shows the correlations" between the Learning Ability 
rating and aptitude tests. Job Knowledge Test, and Work Satopie tasVs for 
.Cartographic Technicians at TOPOCOM installations, again by ethnic r;iter- 
ratee combinations. * , 

'For those technicians rated by Black supervisors, the correlation ' 
coefficients are higher for Black technicians than for Caucasian techni- 
cians on 15 out of the 17 measures. ^Ihe average correlation coefficient 
for.:jBlack technicians is .44 and for Caucasian technicians .22.' 

^ For those rated by a Mexican-American supervisor, the coefficients 
are higher for Mexican-American technicians than for Caucasian technicians 
on 'only three out of the 17 measures. The average correlation coefficient 
for Mexicaii-American technicians is .26 and for Caucasian technicians .42. 

For those rated by Caucasian supervisors, higher validities for the 
Caucasian technicians than for the Black technicians are shown on nine out 
Of the 17 measures, and higher coefficients for the Caucasian technicians 
chan for the Mexican-American technicians appear for only six out of the 
17 me^isures. The average coefficient is .22 for Black technicians, .24> 
for >!exican-Acierican technicians^ and *22 for Caucasian technicians. 

Table VIII-9 shows the correlation between the Learning Ability - * - 
rating and the aptitude tests, for Cartographic Technicians in the Coast 
* a;i<J^Geode*tic Survey sample. For 'these technicians rated by a Black 
supeirvisoir, there were higher validities for Black technicians than for 
Caucasian technicians on seven out of 13* measures. The average 
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coefficient is ♦59 for Black technicians and ,47 for Caucasian techni- 
cians; * •! ^. . > 

*For those rated by a Caucasian supervisor,^, there are higher 
coefficients for the Caucasian technicians than for Black technicians 
on seven out of 13 measures. The average/coef f icient is"»27 for both 
groups of ratees* ^ 

Table VII^IO shows the correlation coefficients between Learning 
Ability rating and (aptitude test scores and Work Sample Overall Perfor- 
mance scores by ethnic group rater-ratee combinations for Inventpry 
Management Specialists. / , - ^ " - 

For those managers rated by a^^Black supervisor, higher cqifficients 

appear for Black managers than- for Caucasian managers on 11 out ^of 15 

I, 

measures ♦ The average coefficient is ;40 for Black managers and .28 for 

• i ^ " ; 

Caucasian managers. For those^* managers rated by* a Caucasian supervisor, 
higher coefficients appear fpr Caucasian managers than for Black manager 
on only two out of 15 measures, and higher coefficients appear for 



Caucasian managers than for Mexican-American managers -on only one out 
i ' • / ^ / ^ 

/ / * 

of 15 measures. The average correlation coefficient is .26 for Black 

managers, .28 for Mexidan-Araerican , managers, and .20 for Caucasian 

managers. 

The comparison of relative/ size of correlation coefficient is 
summarized in Table VIII-11. Black raters appear to base their ratings 
on the qualities measured by"^ the objective instruments to a greater 

degree when rating member^ of their own ethnic group than when rating 

/' 

Caucasians. Other ratei^s appear to have the opposite tendency, t^hat is 
to base their ratings on the qualities measured by the objective 

' S09 . 



instruments to a greater degree when rating members of other ethnic 

groups" than when rating members of their omi group. | ^ 

* . I- "-^ 

Table VIII-12 shows the mean corrielation coefficient (computed by 

r to z transformation) for each of the ethnic group rater-ratee 

combinations by occupation and for/ the overall total. Within each of \ 

the occupational .gi'dups, as well as vfor the totalf, the highest average 

coefficient is for the group of Black ratees rated by Black raters, 

Mexican-American raters were available in only one study, .The 

average correlation for Caucasian technicians rated by Mexican-American 

.supervisors is qu4te high* The average coefficients for the remaining 

^^rpiips, Caucasians rated, by Black supervisors, Mexican-Americans rated 

by Mexicati-American supervisors, and all three ethnic groups rated by 

Caucasian supervisors, are considerably lower and approximately equal. 

Summary 

The evidence in differences in mean ratings, differing relation- 
ships between Job Knowledge Test scores^ and Job Knowledge rattftgs, and 
differing relationships between Learning Ability and^ objective measures 
for the various ethnic group rater-ratee combinations suggests the 
following conclusions: 

1. Raters appear to be more lenient toward members of their own 
ethnic groups. ' - 

. 2. Raters of different ethnic groups appear to base their ratings 
oh different aspects of performance, and these may also vary according 
•CO the ethnic group being rated. 
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Chapter IX ' ^ 

* . ' '* <. 

Factor Analysis of Predictor and Criterion Variables 

* * ^ 

' by Ethnic Group 

The relationships of. single, measurements of aptitude or ability to " 
a single cri'te'rion of job performance, or of a combination of several 
measures of- aptitude to a single; criterion, are important .,in assessing 
.the validity of selection programs. However, these kinds of analyses 
.would liot . necessarily show the existence of different patterns, of ^ - 
"^abilities from one ethnic group to another, if these, patterns ^id in 
rac't exist.. Previous work by Lesser, Fifer, and Clark (19*65), 
StoidTsky and. Lesser (1967) ,, and Flaugher- (19.71), found^ dlKerent 
patterns of mean scores from one cultural group to another. 

'■ To explore th^'possibilit;^ that such differential patterns would 
b'e -found in .the present; data, test, work sample, and supervisors' rating 
data :were factor analyzed separately by ethnic group for each occupation, 
'Selected background variables were added by extension. Principal factor 
solutions were obtained, and for each sample five factors were retained 
and rotatied orthogonally by the yarimax procedure. Adherence to a strict 
numorical^ criterion (eigenvalue of 1-00 or greater), woul<} have called for 
retaining only four factors in several instances, but, it was decided to 
retain a uniform number for all analyses for ease of coiaparisor*. 



/ 

ase of 
Medical, Technicians. 



These analyses, done' separately for Black and Caucasian technician.^, 
contained 28 variables: supervisors' ratings on P^ne scales, peer rati^ig 
on .nine scales, nine aptitude tests, and the Job Knowledge. Test. Age, 
Sex, and Civil Service Salary Grade .were added by extension. For >ot! 
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-X supervisor and peer ratings, the value used for each technician was* 
the average rating; that is, ift,fouf co-workers rated, an individual 
on. Flexibility, " the score used forV^hat scale was the average of 
.,1^^'these four, ratings,. 

The results for Black ^technicians are shown in Table IX-1, 
(Loadings 6f ,2^^ or higher are underlined; loadings of •23 or ,24 
are in- parentheses,) Factor I appears to be primarily a supervisors' 
general opinion factor, shared somewhat by the opinions of corworkers 
^s;shown by the loadings on peer ratings. The loadings on the Job 
Knowledge Test and the Subtraction and Multiplication Test indicate 
that ithe abilities measured^ by these two tests were apparently 
taken into account ,by the supervisor and, to some extent, by the co- 
workers in forming opinions reached in their ratings. Salary Grajjfe 
. also receives a positive loading on Factor I, indicating chat salary 
grade of an individual may influence supervisors' ratings and, to a 
.lesser extent, ratings of co-workers, ^ 

Factor II appears to represent the co-workers', general opinion. 
This factor also has substantial loadings on most of the scales for 
supervisors' ratings, indicating a further sharing of opinion not 
covered in Faptor I, Although, again, the Job Knowledge Test has a 
loading on this factor^, neither the" Subtraction and Multiplication 
. . Test nor any of the other aptitude tests apparently relate to this 
portion of co-worl^rs* evaluations. Salary .Grade also receives a 
positive loading on this factor, ■ ' ^ 

Factor III has its highest loading on the Pin-Dexterity Test, 
and the .next highest on the Gestalt* Completion and Number Comparison 
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tests. In fact, Factor III picks up some loading from all of the aptitude . 

tests other than Vocabulary. This seems to be an analytical function 

• ■ it - 

factor, which has |ieen found in other s,tudies. The high loading on the Ein- 

-Dexterity Test probably is a reflection of the rather complex directions 

- i 

for this test. It should be- pointed out that neither Factor III nor either 
of the remainingjwojactors picks^uf^any loadings on supervisors' ratings 
or co-workers' ratings. . - 

Factor IV has its highest loading on the Necessary Arithmetic Opera- 
tions Test; and its next highest loading on the Vocabulary Test, with 
substantial loadings on Paper Folding,' Picture-Number , Subtrafetion arid 
Multiplication, and Job Knowledge tests. This factor is probably measuring 
Verbal ability primarily, including the ability needed to understand the 
verbal directions on otherwise nonverbal tests. The loadings of this 
factor on supervisors' ratings of Job Knowledge and Learning Ability may 
indicate some real relationship^ other than chance variance, but sucH small- 
loadings sffould be interpreted with caution. The negative loading on Age 
perhaps- Reflects a speededness in this factor, in that with increasing 
age, individuals tend to score lower on speeded^ tests. 

Factor V> with its highest loadings on the Subtraction and Multi- 
plication Test, and tlie only other loadings on Number Comparison and on . 
•Age (negative loading) , may probably best be interpreted as speed of 
numecical operations. . '. > • 

For the Caucasian sample, the loadings shown in Table IX-2 reveal a 
similar pattern. Factor I represents predominantly supervisors' overall 
opinion, with all nine" rating scales receiving quite higl>-'4oadin|;r^ This . 
opinion is also reflected in part in ratings by fe^^kers, with four of 



the rating scales receiving loadings oVef ♦25 and none with a loading 
lower than The Job Knowledge*-Test has a positive loading, but"' 

hone of the aptitude tests has any sizeable loading* Salary Grade 
again receives a . positive loading on this f actor • . 

Factor II reflects predominantly co-wbrker overall opinion, as 
for the Black ^technicians, also shared in part by supervisorsf^with 
loadings of •25. or higher on, six of the nine supervisors' rating 
scales. Salary Grade has a loading of •26, but neither the, Job 
Knowledge Test nor any of the aptitude tests show loadings on. this 
factor, , ' 

Factor III, with highest loadings on the Paper Folding, Gestalt 
Completion, and Pin-Dexterity tests, and substantial loadings on the 
Hidden Figures, Number. Comparison, and Picture-Number tests, probably 
is a measure of analytical functioning. I'Jhis factor also has a 
positive loading on supervisors' rating of Flexibility, which 'indicates^ 
that the abilities measured by tests of spatial visualization, speec^z 
of. closure, and finger dexterity were being considered when supervisors 
evaluated technicians on their ability to shift readily from one 
activity to another • Agelfcreceives a negative loading, again probably 
reflecting speededness of the tests, or perhaps less flexibility in 
shifting from one activity^ to another^ 

Factor IV appears to be a measure of ability to work with numbers. 
The only substantial loadings are those on the ^ubtraction and Multi-^ 
plication. Number Comparison, Necessary Arithmetic Operations., and 
Picture-Number tests. None of the ratings has a substantial loading 
on this factor. 

rr -230 ■ -t'S- 
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Factor V has its highest loadings on the Vocabulary Test, substantial 
loadings on the Necessary Aritlimetic Operations and Job Knowledge Tests, and 
loading to a lesser extent on the Paper Folding Test,. Co-workers' ratings^ 
on Coinmuaication (ability to conmunlcate effectively) and supervisors* rat-r 
ings on :the same scale, have positive loadings. Apparently, Factor V is a 
yerbai .ability factor/ 

the ^.ratings of Black technicians by both supervisors and co-workers are 

♦ if 

related to thd abilities measured by the Job Knowledge Test, and the super- 
yisors* evaluations are related to some extent to the abilities measured by 
the Subtraction and- Multipli^^ but other than that, there seems 

rittle overlap between the abilities measured by the aptitude tests and 
supervisors' and co-workers' evaluation of job performance. 

the ratings of Caucasian technicians by sup.ervisors and co-workers are 
related to some degree to the abilities measured by the Job Knowledge Test# 

Two rating scales. Flexibility for supervisors' ratings and Communicatioiv for 

^, . , ' 

co-workers' ratings, showed some overlap with aptitude test measurement. 

I- . . • '-^ 

On the whole, the structure of the five factors f^r the two groups is 

quite similar. There appears to be little overlap of supervisors' or co- • 

workers' rat ings with aptitude measures, although for both groups the Job 

Knowledge Test, as a performance nsasure, loads'on the rating factors. 

Cartbgrapliic Technicians 

The nekt three analyses show the results for Cartographic Technicians at 

the Army Topographic Command (TOPOCOM). Twenty-five variables were included 

in these analyses: supervisors' ratings on eight scales^ 13 aptttude-irests, 

the Job Knowledge Test, and three work sample tasks. €n additioh,wAge, Sex, 

Salary Grade, and Years of Experience were added by extension. 
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. Table ix-3 shows the rotated factor loadings for Black Cartographic 
Technicians at TOPOCOM installations. 

■a • * 

^ Factor I reflects supervisors' general opinion, with high loadings: 
on ail rating scales, evidence of a general "halo" effect in the ratings. 
Only one of the tests. Civil Service Arithmetic, has a loading which may 
possibly be meaningful- (.23) . Salary Grade has a substantial loading 

C^35). ^ . . 

Factor il has 4its highest loadings bn the Surface Development, 
Civil Service Arithmetic, and Following Oral Directions: tests. The 
/Necessary i^ithmetic .Operations, Identical "Pictures, l&ze Tracing Speed,. 
Card-Rotations,, and Hidden Figures tests also have high loadings. The 
Object-Number Test has a substantial loading as well. The Job Knowledge 
test has a high loading, and the Pull-up work sample task has a substan- 
tial loading. Age has a substantial negative loading and there is also 
a negative loading on Years of Experience, which may be meaningful. The 
learning Ability rating also has a substantial loading. . Again, this may 
be ^n analytical functioning factor, but in view of. the high negative • • 
loading on Age, it may be more accurate to call this factor speed of 
functioning at perceptual and physical tasks. 

Factor III lias its highest loadings on the Vocabulary and Extended 
Range Vocabulary tests. There are also .loadings on the Necessary Arith- 
metic Operations and Job Knowledge tests, as well as on Age and Sex, and> 
to some degree on Years of Experience. This appears to be a verbal 
'ability factor. The positive loading on Sex indicates a slight differ- 
ence on this" factor in favor of women. 

Factor IV has its highest loadings on the Logical Contouring and 
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Pull-up work samplt: tasks • There are also loiidings on the Job Knowledge 
-Test arid^^the Hidden Figures Test, with a possiblymeaningful loading on 
the Maze Tracing Speed Test, Salary Grade and YearS of Experience also 
receive positive loadings. This may be a job,knowledge«'or experience 
•factor, with no apparent differentiation for age, 

;Factor V has Its highest loadings on the Coordination, Ma2e Tracing 
Speed, and Identical Pictures tests, vith a possibly, meaningful loading 
on Map Planning, ' There are negative loadings ^on the Job Knowledge Test 
and Years of Experience. It is possible that this is a spatial ability 
factor, although the absence of loadings on the Card Rotations and 

iSurf ace Development tests argues against that, leading to the tentative 

" '\\ ' ' ' . t 

cqnclusibh that it is an error factor, 

table IX-4 shows the loadings for Mexican-American technicians at 

>* 

TpPOCOM installations, 

'* * . • 

Factor I is again the rater "halo" factor, with substantial loadings 
oh all eight rating scales, ^ It also receives substantial loadings on the . 
job Knowledge Test and, on the Pull-up work sample task, while the loading 
on the Hidden Figures Test is possibly meaningful/ 

.Factor II has its highest loadings on the Necessary^Arithmetic Opera- 
tidns and Surface Development tests, and other substantial loadings on the 
Following Oral Directions, Hidden Figures, Job Knowledge, Map Planning, 
Maze Tracing Speed, Card. Rotations, and Civil Service Arithmetic tests. It 
also has a possibly meaningful loading on the Logical Contouring work sample 
task, and positive loadings on the Leariilng Ability and Interest ratings. 
There are negative loadings on Age, Salary Grade, and Years of Experience. 
This can probably be correctly described as an analytical functioning factor. 
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.'.Factor III has its highest loadings on the Maze Tracing Speed and 
Identical Pictures tests. There are- also substantial loadings on Civil 
Service Aritliraetic, the Pull-up ind Logical Contouring work sample tasks, 
and. the Coordination, Hidden Figures, Surface Development, Map Planning, 
and Job Knowledge tests. There are negative loadings on Age and Years of 
Experience. Perhaps this can be c^ll6d ability at visualizing spatial 
relationships. ^' 

Factor IV Is ^a verbal ability factor. The two vocabulary tests 
receive very high^ loadings, following Oral Directions also receives a 
positive loading. It should be noted that in contrast to earlier analyses 
Age did not receive a po.sitive loading here. Apparently, the older 
•Mexican-American technicians had no advantage over the younger ones on 

testts of English vocabulary. 

Factor V has its highest loadings on the Restitution and Logical 

p 

Contouring work sample" tasks,, and other positive loadings on Card 
Rotations, the Pull-up work sample task, and Salary Grade. There is 
a. negative loading on the Object-Number Test. .Since all three work 
sample tasks had loadings "on this factor, perhaps it could be considered 
a work sample factor. <> 

Table IX-5 shows the loadings for Caucasian technicians at TOPOCOM 

0 

•> 

/* • 

installations. ' o 

. Factor I reflects the supervisors' general opinion, with substantial 
loadings on all rating scales, indicating a substantial halo effect. The 
Job Knowledge Test also receives a substantial loading, as does Salary 
Grade. 

. Factor II has its highest loadings on the Maze Tracing Speed, 
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Identical Pictures, Map Planning, and Card Rotations tests. There are also 
substantial loadings on the Hidden Figures, Civil Service Arithmetic, 
Following Oral Directions, Coordination, Necessary Arithmetic Operations, 
and Job Knowledge tests. The supervisors' ratings on Dexterity also receive 
a substantial loading and there are negative loadings on Age and Years of 
Experience. This may be an analytical functioning factor, but it appears 
more accurate to call it speed of functioning at perceptual and physical tab 

Factor III has its highest loadings on the Pull-up work sample task, and 
the Job Knowledge and Surface Development tests. There are "substantial Joad- 
ings on the Logical Contouring and Restitution work sample tasks^ and the 
Following Oral Directions, Maze Tracing Speed, Card Rotations, Hidden Figures, 
and Necessary Aritihmetic .Operations tests. There, is a posit-ive loading on 
Learning Ability rating. A negative loading on Sex indicates that men do 
better on this factor than do womenT This factor perhaps can be described 
as ability at spatial visualization, although the Joadings on the Following 
Oral Directions and Necessary Arithmetic Opeirations tests, and to some extent 
the loading on the Jol^' Knowledge. Test, are not consistent with this description. 

Factor IV is aga'in the verb\al 'ability factor, wifch very high loadings on 
both vocabulary tests and other loadings on the Job KncW.ledge and Necessary 
Arithmetic Operations tests, as well as on Age and. Sex, pattern previously 
found. 

Factor V has its highest loadings on the Necessary Arithmetic Operations 
and Civil Service Arithmetic tests, with other loadings on the Following Oral 
Directions, Job Knowledge, and Map Planning tests. There are negative load- •* 
ings on Age and Years of Experience. Probably this Ik a numerical ability 
factor, with an element of speededness. 
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In suimnary, for Cartographic Technicians (TOPOCOM) , Factor I, 

representing supervisors* general opinion, and Factor II, an analytical 

functioning factor with a speededhess element, are like those found for 

Medical Technicians, and loading patterns are similar for the three 

ethnic- groups. Factor III is a verbal ability factor for the Black 

sample and a spatial visualization factor for the Mexican-i^erican and 

♦ 

Caucasian samples, while Factor IV appears as the verbal ability factor 
for Mexican-rAmericans and Caucasians and a combination of spatial * 
visualization, job knowledge, and exjierience for Blacks. Factor V, 
which accounts for a very small part of the total variance, differs 
somewhat for the three groups, in that for Blacks the factor represents 
primarily spatial visualization ability, for Mexican-Americans perfor- 
mance on the work sample tasks, and for Caucasians numerical ability, 
because of high loadings on the Necessary Arithmetic Operations and 
Civil Service Arithmetic- tests. There were also loadings fore knowledge 
and experience on this factor .tor Caucasians. 

While some differences are seen in the factor patterns for the 
separate ethnic groups, the differences are not substantial. The 
loadings are quite similar within factors,- \nd reflect essentially 
the same relationships between predictor and criterion variables 
described in earlier chapters. 

The next two analyses show the results for Cartographic Technicians 
at the Coast and Geodetic Survey. There are 21 variables in these 
analyses, including supervisors* ratings on eight scales and 13 aptitude 
tests. Four variables. Age, Sex, Salary Grade, and Years of Experience 
were added by extension. 



Table IX-6 shows the rotateS factor loadings for Black Technicians at 
the Coast and Geodetic Survey. Factor I is the rater halo or supervisors' 
opinion factor, with high loadings on all eight rating scales. This halo, 
or general opinion, appears to be related to performance on several of the 
tests. The Civil Service Arithmetic, Identical Pictures, and Object-Number 
tests have substantial loadings, and the Card Rotations and Surface 
Development tests have loadings which nay be meaningful. Salary Grade also 
has a substantial loading on this factor. 

Factor II has substantial loadings on all of the tests other than the 
two vocabulary tests and* the Object-Number Test. The Vocabulary Test, 
which was administered with speeded conditions, has a loading which^may be 
meaningful. The rating on Learning Ability also has a loading which may 
be meaningful. This appears to be a general, other than verbal, ability 
factor, although perhaps it could be labeled ability at speeded tasks. 

Factor III has its highest loadings on the Hidden Figures and Object- 
Number tests, and substantial loadings on the Card Rotations, Map^ Planning, 
Surface Development, Maze Tracing Speed, and Identical Pictures tests. In 
addition, the Civil Service Arithmetic Test has a loading^which may be 
meaningful. The Learning Ability and Dexterity ratings have substantial 
loadings, and the Job Knowledge rating has a loading which also may be 
meaningful. Age and Sex have substantial negative loadings. This appears 
to be an analytical functioning factor* 

Factor IV has its highest loadings on the two vocabulary tests, and 
other substantial loadings on the Civil Service Arithmetic, Surface 
Development, and Following Oral Directions tests. Age and Salary Grade 
also have substantial loadings. This appears to be a verbal ability 

237 
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factor, but it is not clear why the Surface Development Test should load 
on a verbal factor. 

Eactor V has its highest;, loading on the Coordination Test, and other 
substantial loadings on the Card|^Rotations, Map Planning, Maze Tracing ^. 
Speed, Following Oral Directions, and Identical Pictures tests. It also 
receives a substantial loading on the Need for Supervision rating and a 
substantial negative loading on Age. This appears to be primarily a 
perceptual and spatial relations factor, with an element of speededness 

negatively related to age. 

- Table IX-7 shows the factor loadings for Caucasian technicians at 
the- Coast and Geodetic Survey. Factor T again is the rater halo or 
supervisors' opinion factor, with high loadings on all eight rating 
scales. This opinion is apparently influenced by the abilities measured 
by several of the tests. Th^ Hidden Figures, Civil Service Arithmetic,' 
^Following Oral Directions, and Necessary Arithmetic Operations tests all . 
■^av.e substantial loadings, and the loading on the Surface Development 
Test may also be meaningful. Salary Grade also has a loading, which may 
be meaning fu^l. ' ^ 

Factor^II has loadings on all of the tests except the two vocabulary 

tests. It also has loadings on several of the rating scales. Ratings .of 

/ i 
Learning Ability, Job ^ knowledge. Dexterity, and Need for Supervision all 

have^.substantial loadings, and the loadings on the Overall scale may also 

be meaningful. Age, Salary Grade, and Years of Experience all have sub- 

stantial negative loadings, so apparently the younger technicians are 

better at what this factor is measuring. This probably is the analytical 

functioning factor. 
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Factor Ill^is obvipusly a verbal ability factor, with very high 
loadings on the two vocabulary tests and a possibly meaningful loading 
on the Necessary Arithmetic Operations Test, Age, and Salary Grade 
also have substantial loadings. 

Factor IV has its highest loadings on the Coordination and 
identical Pictures tests, with substantial r^gative loadings on the [ 
Dexterity^ rating, Civil Service Arithmetic, and Necessary Arit^ etic 
Operations tests. There is some possibility that this is an .error 

factor, as it is difficult to explain why those who do best on the 

/ 

Coordination Test should be rated poorly in Dexterity and also do 
pooirly on an arithmetic test, 
- ^Factor V has substantial positive loadiiigs on the Object-Number, 
Civil Service Arithmetic, Map Planning, and Following Oral Directions 
tests,' and substantial negative loadings on the Hidden Figures Test, 
^Age, '^nd Sex, and a .possibly meaningful negative loading on Salary 
Grade, This is most likely an error factor, 

AgainT^'there are some differences .in factor order between^ the 
Black and Caucasian samples. Several, factors' appear to be error 
factors., /in that some of the loadings are difficult to rationalize. 
The sm^llness of the Coast and Geodetic. Survey samples may explain 
some anomalies". 

The last three analyses show the- results' for .Inve'ntory Management 
Specialists. There are 32 variables, in. these analyses, including 
supervisors* ratings* on ten scales, 12 aptitude te^ts, and^ten scores 
on tihe work' sample. Again, Age, Sex, Salary Grade, and Years of 
Experience were added by extension, 

• '239 



^ Table IX-8^ shows the rotated factor loadings for Black Inventory 
Management Specialists. 

. Factor I is clearly. rater ^s overall halo, with all rating scales 
receiving high loadings. The Subtraction and Multiplication Test also 
receives a substantial loading on this factor • 

Factor II is. performance on the Work Sample measure. To some - 
extent, this can be interpreted as a scorer's 'halo act piT, since all 
but one of the Work Sample scx>res received a substantial loading on 
tkis factor. The factor loadings, on several of the Work Sample scales 
r^n considerably lower than the .supervisors' jater halo shown in 
Factor I. thus, it can reasonably be argued that the scorer's halo 
effect is minimal. One test, the Inference- Test, received, a 
subs tantial« loading on this factor, and Salary Grade similarly has a 
substantial loading. 

Factor III has loadings on a number of the aptitude tests. The 
highest loadings are oh the two vocabulary tests, with other high 
loadings on the Federal Service Entrance Examination, the Inference 
Test, and the Necessary Arithmetic Operations Test. Other tests with 
substantial loadings include^Letter Sets, Nonsense Syllogisms, and 
'Following Oral Directions^ One of the Work Sample scores, "Organizes 
^systematically," receives a substantial loading on this factor as 
well. ■ This probably can best be interpreted as a verbal reasoning 
factor, * ' • 

Factor -IV has^ its highest loadings on the Hidden Figures and 
Number Comparison tests, with other substantial loadings pn the 
Following Oral Directions, Letter Sets, Subtraction and Multiplication 



-333- 

* and Necessary- Arithmetic Operations tests, and Federal Service Entrance 
Examination. The Work Sample score, "Productivity," also has a substan- ^ 
' tial loading, as does the rating on. Learning Ability. Age has a 
substantial negative loading. This factor probably can be considered ^ % 

as ability at speeded operations. 



^ Factor V has its highesft. loadings on two of the Work Sample scores, 
"Maintains controls" and "Shows Inventory Manager knowledge." It also 
has a substantial loading on "Tafces problem solving action." The Object - 

.Number- Test and the Subtraction and Multiplication Test also have sub- 
stanti'al loadings, while Age and Years of^perience have substantial 
riegatiye loadings. , This may be an error factor or perhaps might be con- 
sidered memory for details. , ^ - 

Table 'IX-9 shows the rotated- factor loadings for .Mexican-American 
-Inventory Management Specialists. Factor I represents the rater's pver- 
all judgment, or halo^, with very substantial. loadings on all ten rating 
scales. The Nonsense Syllogisms Test a]*^-» has a substantial -loading, 
does the "Analyzes problems" score from the Work Sample. Sex has a 
negative loading, indicating a slight tendency to rate Mexican-American 
males higher. 

Factor II reflects primarily pe rformance on the Work Sample, with 
substantial loadings on all of the Work Sample scores except "Productivity." 
There are. also substantial loadings on several of the aptitude tests, 
including the, Extended Range' Vocabulary, Inference, Federal Service 
Entrance Examination, and Number Comparison tests. 

Factor III has its highest loadings on the two vocabulary tests and 
the Federal Service Entrance Examination. There are also substantial 
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loadings on several other tests, including Inference, Necessary Arith- 
metic Operations, Hidden Figures, Following Oral Directions, Letter 

Sets, Subtraction and Multipl^ication, and Object-Number. Several of u 
^ / ' . 

the Work Sample scores also^^have substantial., loadings on this f attor. 

These include "Organizes ^ystematically," "Quality of action,"- "Takes 

problem solving action,'* "Analyzes problems,". "Follows directions," 

/ • , 

and scorer's rating of overall Work Sample performance. The supervisor's 

^ ^ / ...... . 

rating on effective communication may also be meaningful. This factor 



appears to represent, verbal reasoning ability, primarily, ^ 

/ ' 

' Factor IV/ has its highest loadings on the Number ^Comparison and 

./■ 

Subtraction and Multiplication tests. Th^re are also substantial 
loadings on-^^the Letter Sets, Necessary Arithmetic Operations, Follow- 

1 ■ • 

ing Oral, Directions, and Object-Number tests, and the Federal Service^ 
Entr.ance Examination. There are also substantial loadings on sup'er- 
visqrs' ratinjgs of Learning Ability and, effective Comm :nication, and 
on/ the Work Sample rating of overall performance* There is a negative 
loading on the supervisors' rating of Cooperation and on Age. Sex has 
a positive loading, indicating better performance by women. This factor 
"appears to. represent ability at speeded operations. 

Factor V has its highest loadings on the Nonsense Syllogisms and 
Following Oral Directions tests, and on the Work Sample "Productivity" 
score. The Letter Sets and Necessary Arithmetic Operations tests have 
substantial loadings, as do two additional scores from the work sample: 
"Analyzes problems" and "Maintains controls." Sex has a negative loading. 
This may^be an error factor, or it may possibly represent ability to work 
effectively with somewhat unfamiliar types 'of materials. 
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Table IX-10 shows the rotated factor loadings for Caucasian Inventory 
Management Specialists. Factor I is clearly a rater halo factor, 'with lead- 
ings on each of the ^t&n rating scales. There is also a substantial loading 
on the Subtractio.n and Multiplication Test. 

• ' Factor II is a work sample performance factor, with very substantial 
loadings on all ten scores. There are also substantial loadings on three 
of the aptitude tests: Number Gomparisoh, Following Oral Directions, and 
Vocabulary. - * ^ 

Factor HI: has it s^ highest loadings on the few^ vocabulary tests and 

the Federal Service Entrance Examination. There are also very substantial 



■loadings on a'riumber of the other aptitude tests. These are Letter Sets, 
Inference, Ne^pessary Arithmetic Operations, Nonsense Syllogisms, and 
Hidden Figures. None o^ the rating scales or work sample scores has any 
substantial loading on this factor, but Years of Experience has. a substaii- 
tial negative loading." This factor apparently r^^presents primarily verbal 
reasoning ability. 

Factor IV has its highest loading on the Number Comparison Test, and- 
the next highest loailfng on the Subtraction and Multiplication Test. There 

were also substantial .loadings on the Letter Sets, Hidden Figures, Necessary 

>* 

Arithmetic Operations, and Following Oral Directions tests. Age has a sub- 
stantial negative^loading. Apparently this factor represents abili-ty at 
speeded operations. 

Factor V has its highest loading on the Object-Number Test,* and a sub- 
stantial loading on the Subtraction and Multiplication Test. There is a 
negative loading on the. Hidden 'Figures Test and on the "Productivity" Work 
Sample score. Quite possibly this is an error , factor , since there does 
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1 

not appear to be .a logical explanation for these relationships. 

For Inventory Management Specialists, the first four factors show 
quite uniform patterns for the. three ethnic groups. The fifth factor 
appears to be an etror factor, since relationships of variables load- 
ing on this factor are not readily iriterpretable for any of the three 
ethnic "groups. On the whole, however, the factor patterns are quite 
similar across ethnic groups - 

^ ' ' . Summary and Conclusions * * . 

In general, differing patterns of abilities are not readily dis- 
cernible from^one^ethnicgrpup to another in the three occupations 
studied* Factor structures appear to be quite 'similar across occu- 
pations and ethnic groups, .With a few exceptions, abilities measured 
by the aptitude tests do not appear to have been taken into account 
in ratings by supeirvisors of various aspects of job performance. 
However, supervisors do appear to be reflecting in their ratings to 
■some degree aspects of job performance measured by the Job Knowledge 
Test and Work Sample, Various aptitude tests are related to the 
latter criterion measures, to a much greater degree than to ratings. 
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Table IX-1 

Factor Analysis of Predictor and Criterion Variables 



Medical Technicians 



Black Sample*^ 



Rotated Loadings 



Variables Included in Factor Analysis 



Supervisors* Ratings 

Flexibility 
Organization . 
Interest 

Learning Ability 
Job 'Knowledge 
Technique 

Need for Supervision 
Cdotmunlcation * 
Overall^ ^ — 

Pe^r Ratings 

iPlexlblllty 

Organization 

Interest 

Learning Ability 
Job Knowledge 
Technique 

Need for Supervision 

Communication 

Overall 

Aptitude Tests 



I 


II 


III. 


IV 




80 


27 


10 


06 


11 


82 


29 


. 02 


-03 


13 


80 


19 


-03 


' -01 


10 


83 


33 


;i7 


19 


• 06 


,85' 


22 


% 


22 


.01 


84 


30 • 


04« 


' 06 


04 


80 


31 


00 


07 


02 


78 


22 


03 


16 


-06 


85 


27 


05 


-03 


06 



21 


76 


' 07 


-02 


04 


(24) 


79- 


-05 


05 


12 


18 


76 


00 


-02 


08 




78 


09 


05 


•02 


32 


77 


02 


13 


-06 


(24) 


83 


03 


04 


■ -04 


26 


79 


04 


03 


01 


27 


67 


03 


14 


06 


21 


84 


-01 


-02 


-02 



Subtraellon & Multiplication 


26 


08 


. (24) 


28 


71 


Vocabulary 


13 ■ 


-03 


10 


46 


06 


Hidden Figures 


04 


-11 


35 


12 


-03- 


Necessary Arithmetic Operations 


19 


03 


40 


60 


16 


Pin-Dexterity 


08 


-03 


87 


-13 


.16 


Number Compl'etion 


09 


*10 


62 


10 


31 


Gestalt Completion 


03 


07 


- 64' 


22 


-03 


PicturerNumber 
Paper Folding 


-15 
00 


17 - 
12 


30 
48 


33 
"34 


09* 
-07 * 


>b Knowledge Test 


'45 


28 


26 


41 


04 


Sum of Squared Loadings 


6.98 


6.26 


2.36 , 


1,30 


.76 



Variables Added by Extension 

Salary/iSrade 
Sex / 

Age/'' 



35 
-07 
-04 



05 
-10 



02 
• -16 
(-24) 



19 
13 
-28 



-05 
08 
29 



Sample size " 168 
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Table IX-2 

Factor Analysis of Predictor and Criterion Variables 
Medical Technicians 



Caucasian Sample 



Rotated Loadings 



Variables Included in Factor Analysis 



Supervisors^' Ratings 

Flexibility 
Orgaiiization 
interest '^^ 
Learning Ability 
Job Knowledge • 
tecimique 
/Ne'ld for Supervision 
Communication 
-Overall ■ 



%9 



Peer '.Ratings 

Flexibility^ 
Organization 

Interest #f 
Learning Ability 
Job Knowledge 
Technique 

Need for Supervision 
CQcmunication 
Overall . 

Aptitude Tests o 

Subtraction & Multiplication 
.Vocabulary— 
^Hidden Figures 
Necessary Arithmetic Operations 
Pin-Dexterity 
Number Comparison 
^ Gestalt Completion 
Picture-Number 
Paper Folding 

Job knowledge Test 

Sum of Squared Loadings 

Variables Added by Extension 

Salary Grade 
Sex 

Age ' . 

Sample size = 297 



ERIC 
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i 


II 


III 


IV 


It 

V ■ 












72 


(23) 


■'25 


17 


■*03 ■ 


79 


32 ' 


07 . 


07 


-06 


79 . 


17 


03 • 


. 09 


00 • ■ 


80 


25 


22 


15 


06 


75 


, 31 


00 


-05 


19 


77 


34. 


11 


02 


00 


85 


25 




-03 


03 


73 


(23) 


00 


-04 


22 


82 


32 


05 

• 


04 


01 

]^ 


19 


77- 


07 


15 


-01 


21 


81 


-02 




-03 


31 


69 


-03 


09 


-01 


27 


75 


17, 


13 


07- 


("21) 


7T 


08 


00 


11 


(23) 


80 


-03 


04 


-02 


26 


80 


-02 • 


-09 


05 


(24) 


67 


01 


07 


■ 25 


26 


82 


03 


02 


-06 



08 


08 


^1 


72. 


11 


02 


06 


09 


02 


67 


■03 


-04 


55 


10 


11 


11 


03 


60 


33 


39 


09 


07 


68 


17 


-12 


t02 


05 


49 


50 


-06 


06 


01 


72 


-01 


-03 


08 


15 


28 


31 


03 


09 1 


03 


73 


-05 


(23) 


29 1 


06 


19 


19 - 


33 


6.17 


5192 


2.73 


1.19 


.99 



32 

-01 
-06 



26 
00 
03 



-22 
04 
-34 



-11 
08 
(24) 



16 
-03 
-20 
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Table IX-3 



Factor Analysis of Predictor and Criterion Variables 
Cartographic Technicians (TOPOCOM) 



\ Black Sample 
Variables Included in Factor Analysis 

Supervisors^ Ratings 

Accuracy 
' Interest 
Learning Ability 
Job Knowledge 

* Dexterity 
Need for Supervision 
Perseverance 
Overall k 

Aptitude Tests 

Coordination 
"Hidden Figures 

Vocabulary 

Object-Number 

Card Rotations 

CS Arithmetic . 

Map Planning 

Surface Development 

Maze Tracing Speed 

Following Oral Directions 

Identical Pictures . 

Extended Range Vocabulary 
' Necessary Arithmetic oi)erations 

Work Sampl e 

• Restitution 
Logical Contouring 
Pull-up 

Job Knowledge Test 

Sum of Squared' Loadings 
« 

Variables Added by Extension 

Age 
Sex 

Salary Grade 
Years of ^cperience 

Sample size = 100 



Rotated Loadings 



I 


II 


• 


IV 


V 


Q1 




-01 


04 


08 


OH 


05 


11 


-07 


-06 


Oj 




00 


12 ■ 


• 04 






07 


i6 


-06 


82 


09 


-06 


-01 


19 


.91 


1? 


07 


11 


• 00- 


82 


- -02 




-06 


00 


92 


15. 


05 


• 08 


00 


06 ■' 


18 


-12 


05 


■ 57 


13 


■ 53 '. 


■ 19 


26 


03 


13 


16\ 


87 . 


14 


10 


16 


43 


02 


-15 


10 


09 


58 


-05 


. 19 


14 


(23) 


73' 


07 


' 17 


-15 


09 


68 


09 • 


16 


(23) 


15 


75 


10 


' 12 


-03 


01 


■'55 


-03 


\'(23) ' 


38 


09 


71 


20 


-06 


02" 


10 


"66 


-05 


17 


35 


09 


22 


. 88 


08 


-10 


13 


69 


40 


09' 


-03 



03 


18 


13 


21 


03 


03 


03 


14 


65- 


10 


06 




-06 




-01 


15 


61 


31 


39 


-33 


6.29 


4.92- 


2.02 


1.31 


.89 


-04 


-50 


53 


01 


-08 


04 


-01 


27 


-19 


-02 


50 


-11 


- 18- 


33 


09 


07 


(-24) 


(24) 


44 


-34 
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Table IX-4 

Factor Analysis of Predictor and Criterion Variables 
Cartographic Technician^ (TOPOCOM) 
Mexican-American Sample 



Variables Included in Factor Analysis 

Supervisors ' Ratings 
Accuracy 

Interest , . 

learning Ability 
Job Knowledge 
Dexterity 

Need for Supervision 

Perseverance 

Overall 

Aptitude Tests 

Coordination 
Hidden Figures 
Vocabulary 
Object-Number 
.Card Rotations 
CS Arithmetic 

Map Planniiag 
Surface Development 
Maze Tracing Speed 
Following Oral Directions 
• Identical Pictures 

Extended Range Vocabulary 

* Necessary Arithmetic Operations 

Work Sample 

Restitution 
Logical Contouring 
^ Pull-up 

^ Jbb Knowledge Test 

Sum of Squared Loadings ^ . 

Variables Added by .Extension 

Age 
Sex 

Salary Grade 

Years of Experience 

Sample size = 100 



^ Rotated , Load ings 



i 


T T 

1 J 


TTT 

IJLl. 


TV 

X V 


V 


92 




03 


-05 


09 




(23) 


05 


06 


-08 


90 


. 30 


17 


00 


02 


93 


12 • 


06 


-04 • 


06 

• 


84 


06 


22 


-07 


09 


92 


,11 


10 ' 


03 


14 


83 


03 


,12 ^ 


11 


-13 


94 


10 ■ 


06 


-01 


09 


06 


10 


46 


03 


-03 


(23) 


60 


36 


-06 


02 


04 


12 


05 


84 


00 


03 


17 


13 


-05 


-31 


"03 


34 


47 


08 


31 


14 


38 


51 


08 


-01 


20 


44 


26 


-05 


13 


09 


70 


31 


07 


05 


04 


43 


75 


-05 


03 


12 


68 


15 


25 


-13 


10 


28 


72, 


01 


-07 


04 


15 


-02 


82 


04 


14 


71 


22 


17 


05 


09 


35 


08 


03 


43 


19 


(23) 


33 


-12 


46 


35 


10 


49 


-05 


28 


35 


71 


22 


17 


05- 


6.94 


3.31 


2.71 


1.58 


.82 


-13 


-39 


-40 


14 


08 


-07 


-21 


-14 


11 


-07 


.22 


-32 


05 


-01 


33 


-10 


-36 


(-24) 


-08 


16" 
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V 



Factor Analysis of Predictor and Criterion Variables - 
Cartographic Technfcidns (TOPOCOM) 
Caucasian Sample 



Var i ab les Included in Factot Analysis 



Supervisors* Ratings 

Accuracy 
Interest 

Learning Ability 
Job Know.ledge 
Dexterity * 
Need for Supervision 
Perseverance 
Overall 



Rotated Loadings 



Aptitude Tests 

Coordination 

Hxdden Figures 

Vocabulary 

Object-Number 

Card Rotations 

CS Arithmetic 

Map Planning 

Surface Development 

Maze Tracing Speed 

Following Oral Directions 

Identical Pictures 

Extended Range Vocabulary 

Necessary Arithmetic Operations 

Work Sample 



T 
1. 


II 


III 


IV 


92 


14 


06 


07 


85 


04 


14 


05 


84 


19 


25 


-09 


92 


06 


17 


05 


79 


28 


05 


-05 


89 


13 




-02 


81 " 


00 


-01 


11 


92 


17 


08 


-06 


13 


39 


-02 


-03 


11 


48 


27 


^ -1? 


04 


04 


16 


84 


01 ^ 


30 


-01 


19 


12 


56 - 


39 


-15 


14 


45 


-01 


14 


19 


62 


15 


00 


11 


53 


50 


03 


12 


68 


32 


-12 


13 


42 


42 


19 


01 


66 


20 


00 


-04 


-05 


21 


^ 85 


09 


30 


31 


(23) 



Restitution 
Logical Contouring 
Pull-up 


11 
03 
14 


14 
10 
10 


38 
42 
58 


13 
06 
06 


Job Knowledge Test 


26 


27 


54 


30 


Si?m of Squared Loadings 


6.29 


3.12 


2.10 


1.79 


Variables Added by Extension 










Age 
Sex 

Salary Grade 

Ysars of Experience 


05 

-17 
49 
• 17 


-52 
-11 
-16 
-42 


-09 
-37 
06 
10 


41 
34 
08 
19 



Sample size = 240 



249 
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T^ble IX-6 

Factor Analysis of Predictor and- Criterion Variables 
Cartographic Technicians (Coast & Geodetic Survey) 

Black Sample 




Variables Included in Factor Ana 

Supervisors' Ratings 

Accuracy 
Interest 

Learning Ability % • 

Job Koatfledge ^ 
Dexterity 

Need for Supervision 

Perseverance 

Overall 

Aptitude Tests 

Coordination 
Hidden Figures 
Vocabulary 
Object-Number ' 
Card Rotations 
CS Arithmetic 
■ Map Planning 

Surface Development 
Maze Tracing Speed 
Following Oral Directions 
Identical Pictures 
Extended Range Vocabulary 
.^ecessary Arithmetic Operations 

Sum of Squared Loadings 
Variables Added by Extension 



Age 
Sex 

Salary Grade 

Years of Experience 

Sample size =38 



Rotated Loadings 
II III IV 



h 


13 


18 


05 


93 


15 


11 


00 


83 


(23) 


28 


19 


92 


U 


(24) 


04 


86 


02 


29 r 


12 


88 


07 


10 


13 


90- 


16 


-02 


" 04 


93 V 


21 


12 


02 



08 


33 


-04 


10 


14 


31 


71 


22 


02 


(24) - 


■ 10 


87 


29 


02 


73' 


06 


(23) 


53 


45 


15 


37 


63 


(24) 


29 


22 


51 


32 


03 


(24) 


43 


58 


34 


-12 


64 % 


41 


-05 


20 


70 


04 


. 44 


31 


38 


54 


-08 


13 


15 


17 


91 


17 


80 


06 


21 


7.02 


3.28 


2.53 


2.19 



18 


■ -08 


-56 


35 


01 


-10 


-48 


10 


52 


-02 


^ 06 


27 


06 


-17 


-22 


-15 
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0 
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Table. IX-7 



Rotated Loadings 



Factor Analysis of 'Predictor and Criterion Variables 
Cartographic Technicians (Coast & Geodetic Survey) 
Caucasian Sample 
Variables Included in Factor Analysis 

Supervisors' Ratings 

Accuracy . 
Interest 

Learning Ability 
Job Knowledge 
Dextef ity 

Need for Supervision 
Perseverance 
Overall 

Apti^de Tests 



Coordination 
Hidden Figures 
Vocabulary 
Object-Number 
Card Rotations 
CS Arithmetic 
Map Planning 
Surface Development 
. Maze Tracing Speed 

-Following Oral Directions 
Identical Pictures 
Extended Range Vocabulary 
Necessary Arithmetic Operations 

Sum of Squared Loadings 

'Variables Added by Extension ^ 

Age 
Sex 

Salary Grade ' 
Years of Experience 

Samole size =. 5l . 



I 


II 


III 


IV 


90 


18 


01 


-14 






04 


20 




40 


-08 


-03 




2^ 


12 


02 


69 


40 ' 


00 


■ -28 


■79 


41 


-04 


-13 


78 


01 


01 


18 • 


93 


(23) 


01 


-09 


-01' 


32 


07 


50 


.27 


73 


-09 


01 


09 


UU 


OO 


yjo 


-01 


25 


10 


-04 


18 . 


68 


03 


03 


28 


65 


19 




16 


75 


05 


07 


(23) 


. 79 


07 


20 


18 ■ 


74 


-11 


15 


29 ' 


76 


12 


-05 


22 


75 


-22 


(24) 


-06 


■ 04 


88 


-04 


31 


21 ' 


(23)- 


(-24) 


6.04 


5.65 


1.77 


.83 


-16 


-66 


27 


02 


^11 


-10 


22 


11 


(24) 


-39 


26 


10 


-03 


-38 


■ 13 


06 
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Tabl.e IX-8 ' . 

Factor Analysis of Predictor and Criterion Variables 
Inventory Management Specialists 
Black Sample 
Variables Included in Factor Analysis - 

I 



. Rot^ated Loadings 
II • III IV 



ERIC 



Supervisors' Ratings 

Organization , 
Interest 

Learning .Ability 
Communication 
Technical Knowledge 
•Stability 
Dependability 
Practical Judgment 
Cooperation 
Overall 

' Aptitude Tests 

• Number Comparison 
Hidden Figures 
Vocabulary 

Object-Number 7 
^Letter Sets . 
Nonsense Syllogisms 

r Subtraction & Multiplication" 
Extended Range Vocabulary 
Necessary Arithmetic Operations 
Following Oral Directions 

— -Inference 

FSEE . ^ 

Work Sample 

Takes Problem Solving Action 
Uses New Procedures 
Shows I. M. Knowledge 
Analyzes Problems 
Organizes Systematically 
Maintains Controls 
Follows Directions 
Productivity 
Quality of Action^ 
^ Overall Performance - / 

Sum -of Squared Loadings 
Variables Added by Extension 



Sex 
Age 

Sala^ry Grade 

Years of Experience 



87 


12 




0 n 
/I 


no 


88 


07 


07 


11 


nA 


81 


19 


12 


25 


-00 


$2 


04 


06 


Uo 


m 
ux 


87, 


10 


^ no 
08 


06 


-01 


Q C 
03 




HA 




08 


85 


10 


13 


. 07 


13 


89 


'15 


11 


08 


-00 


66 


15 


-02 


-03 


(24) 


90 


08 


10 


16 


08 


21 


09 


07 


69 


11 


17 


13 


22 


70 


-10 


03 


17 


87 


04 


05 


-04 


-08. 


10 


(23) 


. 50 


20 


11 


60 


49 


13 


06 


21 ~ 


65- 


-08 


-01 


29 


-03 


13 


46. 


25 


-09 


16 


91 , 


-03 • 


-06 


21 


15 


71 


32 


08 


08 


(24) 


53 


54 


15 


13 - 


26 


81 


01 


15 


19 


17 


83 


26 


11 



Sample size = 114 



11 


73 


21 ' 


01 


27 


10 


88 


20 


12 


21 


19 


56 


04 


00 


63 J 


10 


71 


22 


-00 


-12 


10 


73 


31 


15 


-09 


14 


(23) 


09 


03 


74 


13 . 


83 


05 


(23) 


-04 


07 


41 


06 


43 


14 


17 


87 


22 


02 


10 


16 


90 


17 . 


06 


10 


7. '59 


5.59 


4.97 


2.40 


X.64 


-09 


-02 


-13 


16 


13 


-19 


08 


13 


-21 


-2k'- 


-03 


25 


-09 


-16 


-12 . 


05 


07 


-14 


-06 


-29 
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Table 'lX-9 

Factor Analysis of Predictor and Criterion Variables 

Inventory Management Specialists 

Mexican-American Sample 

ariables Included, in Factor Analysis Rotated Loadings 

. ■ • I II . Ill . IV 
upervisors Ratings o ., 

■\organization 86 -02 ' 10 07 

\ln teres t , 89 - -00 -01 , -03 

• Learning Ability ' 81 09 15 37 

Conmunication 2A ^24) 34 

Technical Knowledge • 75 01 17 13 

Stability 89 03 04 r-02 

'pependability 84 09 ^15 . -10 

Practical Judgment 85^ 14 ^06 11 

Cooperation ■ . 63-' 21 '10 -28 

Overall 90 , -04 10 02 

Aptitude Tests ♦ 



Number Cbmparison 


10 


26 


03 


67 


Hidden Figures 


. 21 


V 03 


58 


' 18 


Vocabulary 


-05 


19 


80 


04 


■Object-Number 


-06 


12 


26 


29 


Letter Sets 


-09 


14 


38 


58 


Nonsense Syllogisms 


34 


21 


• 40 


,-705 


Subtraction & Multiplication' 


20 


19 


31 


63 


Extended Range Vocabulary 


01 


40 


76 


01 


Necessary Arithmetic Operations 


16 


29 


59 


38 


Following Oral Directions 


20 


13 


■ 45 


28 


Inference < 


- 06 


35 


65 


12 


-FSEE " - 


14 - 


32 


76 


• 25 



Work Sample 



Takes Problem Solving Action 


15 


68 


34 


05 


Uses New Procedures 


06 


88 


22 


13 


Shows I. M.' Knowledge 


-06 


85 


03 


■ 11 


Analyzes Problems 


33 


40 


30 


18 


Organizes Systematically 


22- 


57 


48 ■ 


-14 


Maintains Control 


-03 


^ 68 


-00 


05 


Follows Directions 


13 


80 


27 ■ 


16 


•Productivity 


02 


17 


11 


14 


Quality of Actions 


. 04 


86 


35 


18 


Overall Performance 


08 


81 


32 


29 


Sum of Squared Loadings 


7.22 


5.77 


4.54 


2,23 



Variables Added by Extension 

Sex 
Age 

Salary Grade 

Years of Experience 



-25 


18 


-04 


29, 


-22 


-08 


(-23) 


-42 


14 


(23) 


22 


(24) 


04 


12 


15 


04 



Sample size * 72 
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Table IX-10 

Factor Analysis\pf Predictor and Criterion Variables 
Inventoi;y Management Specialists 



Caucasian Sample 
Variables Included in Factor Analysis. 



Supervisors* Ratings 

Organization ^ 
* Interest 

Learning Ability^^ 

Communication 
. Technical Knowledge 
' Stability 

Dependability 

practical Judgment 

Cooperation 

Overall 

Aptitude Tests 

Number Comparison . 

Hidden Figures 
. Vocabulary 

Object-Number 

Letter Sets q 

Nonsense Syllogisms 

Subtract'iort & Multiplication 

Extended Range Vocabulary 

Necessary Arithmetic Operations 

Following Oral Directions 

Inference . 

FSEE 
Work Sample 

Takes Problem Solving Action 
Uses New Procedures 
Shows I. M. Knowledge 
Analyzes Problems • 
Organizes Systematically 
Maintains Controls 
Follows Directions 
Productivity 



Rotated Loadings 



I 


II 


III 


IV 


V 


86 


13 


-01 


08 


10 


85 


09 


■ ,04 


06 


-02 


85 


19 


09 


09 


10 


81 ' 


09 ■ 


15 


18 


-01 


86 


12 


■ 07 


. 11 . 


01 


'88 


. 09 


07 


08 


' 02 


86 


■ 07 


06 


01 


03 


90 


09 


19 


02 


02 


66 


-11 


09 


01 


-20 


91 


12» 


07 


06 


04 



19 


28 


17 


74 


I 03 


03 


15 


50 


39 


-A] 


04 


26 


79 


-21 


00 


03 


04 


18 


20 


69 


13 


14 


70 


45 


02 


05 


-02 


61 


04 


11 


26 


12 


07 


69 


26 


08 


19 


84 


-22 


02 


17 


16 


71 


39 


01 


06 


28 


58 


35 


-11. 


13 


17 


78 


07 


06 


14 


■ 22 


87 


15 


• 07 



QuaI±ty-of-Actions_ ^ ■ 

Overall Performance 

* Sum of Squared Loadings 

Variables- Added by Extension 

Sex 
Ajge 

Salary Grade 

Years of Experience 

Sample size = 200 



-01 


, 75 


20 


05 


(24) 


10 


89 


18 


11 


13 


11 


■78 


01 


05 


13 


04 


76 


20 


'-07 


-11 


18 


73 


19 


16 


-02 


14 


71 


-03 


04 ' 


-01 


11 


• 83 


.15 


20 


06 


■ 02 


66 


lo 


17 


-28 


08 


■ 89 


(23) 


OS' 


-02 


-1-1- ^ 


-88 


^(23)- 


04 


-12 


7.48 


6.83 


5.10' 


2.01 


1.01 



09 


17 


-09 


(24) 


20 


-17 


-16 


-22 


-35 


-19 


21 


-02 


-15 


-04 


-02 


2^ 


« 04 


-39 


-07 


04 



Chapter X 

« 

Comparisons of Background and Task Variables 
' By Occupation by Ethnic Group 
As described in Chapter II, a comprehensive personal history ques- 
tionnai-re was incl^uded in the test battery for each x)ccupation studied/ 
The questionnaire for each occupation also contained a detailed task 
list. -The purpose "was. to -explore possible differences in composition - 
of the ethnic samples, to relate background variables to measures of 
aptitude and job performance by ethnic group, and to determine whether 
there were variations in the type of work performed by members of 
different ethnic groups. 
Composj-tion of Ethnic Samples 

• Tables X~l (Medical Technicians) , 'X-2 (Cartographic Technicians), 
and }?-3 (Inventory Management Specialists) show the distribution of 
selected background variable'^ expressed as percent of total sample, by 

ethnic 'group. As will be seen, the variables selected for comparison 

* ft 

are those considered most likely to be moderators of job performance: 
age, sex, education, source and. length of training on and off the job, 
source and amount of experience on and off the job, and present GS-level. 

For Medical Technicians, computations of chi-square on Black and 
Caucasian distributions of selected variables show significant differ- 
ences at the .01 level for Age (Caucasian technicians tended to be older 
on the average). Source of training (more Blacks had training in an . 
accredited school and in government hospitals while more Caucasians had 
training in military service and in civilian hospitals). Length of time 
employ.ed at present Installation (71 percent of Blacks employed four 
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years or longer versus 55 percent of Caucasians), and Total years of 
experience as A Medical Technician (57 percent of Caucasian technicians 
had 12 or more* years of experience versus 43 percent of Black techni- 
cians), Chi-square for Total years in Civil Service as a Medical 

Technician was significant at the .#05 level (23 percent of Caucasian 

■ I 

technicians had 16 or more years of service versus 14 perceht of the 

4 

Black technicians) • Chi-squares for Sex > Amount of education ^ Length 
of training s and Salary grade. level were not significant • ^ 

For Cartographic Technicians, F-ratios 'at the .01 level of signifi- 
cance were found for Amount of education (Blacks tended to have more 
education than Mexican- Americans and Caucasians: 61 percent of Blacks 
had 1 or 2 years college or more versus 28 percent of Mexican-Americans , 
and 31 percent of Caucasians); Amount of training prior to becoming 
Cartographic Technician (54 percent of Mexican-Americans had a moderate 
amount of training or more versus 25 percent of Blacks and 22 percent 
of Caucasians); Years of experience as Cartographic Technician prior to 
1371 series (31 percent of Mexican-Americans had 12 or more years experi- 
ence versus 17 percent of Blacks and 16 percent of Caucasians). F-ratios 
significant at the .05 level were found for Sex (62 percent Black males 
and 79 percent Mexican-American males versus 34 percent Caucasian males); 
and Total years of experience as/ Cartographic Technician in 1371 series 
(71 percent of Mexican-Adericans had 8 or more years in-i371 series, 58 
percent of Blacks, and 46 percent of Caucasians). F-ratios for Age, 
Source of formal training , Lenj^th of training on the job , and Present 
salary grade were not significant. 

For Inventory Management Specialists, chi-squarei values were 



computed. Significant differences at the •01 level were found for' Sex 
(62 percent male Mexican-Americans, 52''percent male Caucasians, and 37 
percent male Blacks); Age (30 percent Black and 39 percent Caucasian 
inventory managers were 50 years old or older versus 11 percent of 
Mexican-Americans); Education (51 percent of the Blacks had 1-2 years 
of college or more versus 44 percent Mexican-Americans and 33 percent 
Caucasians); Length of time^at present installation (85 percent of 
Mexican-Americans had been employed at present installation 8 years or 
longer versus 68 percent Blacks and 45 percent Caucasians) ; and Present 
salary grade (8 percent Mexican-Americans in Grade 11 versus 21 percent 
of Blacks and 20 percent of Caucasians, and 19 percent Mexican-Americans 
in Grade 7 versus 6 percent Blacks and 4 percent Caucasians). Chi-squares 
for Amount and Source of training ^ Length of training^ and Total years of 
experience were not significant. 

Differences in Distribution of Salary Grades Across Ethnic Samples 
Some differences in the distribution of the ethnic samples bjr 
Present salary grade (GS-level) were found. To determine whether these 
distributions are significantly different, chi-square values were 
computed separately by ethnic group versus each other ethnic group for 
this variable for all three occupations. For Medical Technicians, 
chi-square was not significant, as noted above. For Cartographic 
Technicians, chi-square was not significant for Blacks versus 
Caucasians, but was significant at the .01 level for Mexican-Americans 
^^ersus the total .Caucasian sample. However, when the sample of Cauca- 
sians was subdivided into those at- San Antonio and those at all other 
installations (see Chapter VIII for other such comparisons) chi-square 
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was ^not significant for differences between the samples of Mexican- 
Americans and Caucasians at the San Antonio djnstallation. For the 
Inventory Management Specialist samples, chi-square was not signifi- 
cant for Blacks versus Caucasians.. Chi-square was significant, at 
the .01 Ipvel, for Mexican- Americans versus all Caucasians, and when 
,the distribution f or^Caucasians at San Antonio was compared with the 
Mexicans-American sample (all from San Antonio), the difference was. 
'Significant at the .05 level. The difference is at the GS-7 level. 
A few more Mexican-American Inventory ^lanagers at the GS-7 level had 
been included in the study than Caucasian GS-7's at .>an Antonio,, to 

^ - I 

increase the ethnic Sample size. The effect of this difference on j 
. aptitude and performance measures may be, observed in later comparison^'. 
Relationship of Selected Background Variables to Aptitudgf and Job 

Performance Measures ^ ^ 1* 

/ ' I 

In this "section, relationships of selected background variables/ 
to aptitude and job performance measures will be ^.discussed where they 
aVe significantly different from zero, by ethni'c group and^ by occupa- 
tion. More detail will be provided in tabled form for correlations of 
present salary grade (GS-level) with aptitude tests and criterion | 
measures because of differences described in the previous section and 
because factor analysed presented in Chiaptet IX showed loadings of^ 
salary grade on factors on which aptitude and job performance measures 
also loaded. 

— -For^Medical_Technicians correlations of selected background ^vari- 
ables^ with aptitude and job performance measures for the two ethnic 
^groups had no notabie patterns of difference. Table X-4 shows Jhe 

\ 
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correlations of Present salary grade (GS-level> Xi^ith aptitude test 

scores, selected supervisors* ratings, and Job Knowledge Test score* 
r 

Significant correlations are seen for present GS-level for both Blacks 
and- Caucasians with supervisors' Learning -Ability and Job Knowledge 
ratings and Overall Rating, and with Job Knowledge Test (in each 
case higher for Blacks). A few positive and significant correlations 
of present GS-level with aptitude tests appear for Blacks. The 
correlations of present GS-level with aptitude tests for Caucasians 
are .in general negative and' low. 

For 'Cartographic Technicians, some trends may be noted for certafin 
other background variables. In general, Age correlates positively with 
Vocabulary test scores, but negatively with some aptitude test scores 
for all ethnic groups. Time in grade level also tends to correlate 
negatively and significantly with measur^of ability (bo^h ratings and 
certain aptitude test scores) for all ethnic groups. For Mexican- 
Americans, a significant and positive relationship is shown for speak- 
ing knowledge of English with those aptitude tests where reading or 
language comprehension is an element. This effect was discussed in a 
previous chapter, where Mexican-Americans were found to score lower on 
vocabulary tests. (Almost all Mexican-Americans indicated that a 
language other than English was spoken in their parents' homes.) Table 
X-5 shows the correlations "of Present salary grade (GS-level) with 
selected aptitude test scores and'^job performance measures for Carto- 
graphic Technicians by ethnic group. For the Black and Caucasian 
samples," as also shown, for Medical Technicians, GS-level relates 
positively and significantly (at the .01 level) with selected supervisors 
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ratings, indicating that those in the higher GS-levels were rated higher 
on Learning Ability, Technical Knowledge, and Overall job performance. 
The correlation of present -.salary grade with Work Sample Composite was 
significant for the Black sample only (at the .05 level)* The only 
significant relationship of present salary grade with any aptitude test 
score was found with the Vocabulary Test for the Black sample.' Present 
salary grade correlated significantly only with^ supervisors' rating of 
Technical Knowledge for Mexican-Americans. 

For Inventory. Management Specialists, as found for the other two 
occupations. Age tended to correlate negatively with measures of learn- 
ing ability (both aptitude test scores and supervisors' rating of . 
.earning Ability) for all three ethnic groups.- Level of education 
attained, on the -other hand, was correlated positively and significantly 
with scores on the Federal Service Entrance Examination for all three 
ethnic groups. In addition, for Mexican-Americans, Standing in high 
school graduating class was correlated positively and significantly 
with Vocabulary Test scores and Highest level of math" used on job with' 
Subtraction and Multiplication Test scores. . . ^ 

Table X-6 shows the correlations of Present salary grade (GS-level) 
with selected aptitude test scores and job performance measures for 
Inventory Management Specialists. The pattern here is quite different 
from that* shown for Cartographic Technicians. The only significant, 
relationship of GSrlevel for Blacks is with Work Sample Overall Perfor- 
mance (at the .05 level), and for the Caucasian sample, with 
supervisors* Technical Knowledge and Overall ratings (at the .05 level). 

■ i 

Correlations of GS-level with aptitude test scores for both ^Blacks and 
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Caucasians' are low, usually negative, and non-significant. For* Mexican- 
Americans, those at higher GS-levels tended to receive higher ratings on. 
Learning Ability and Technical Knowledge and to perform at a higher devel . 
on the Work Sample, They also tended to score higher on most aptitude 

* tests, including the FSEE, implying a significant relationship between 
abilities and- present level on the job. ' . 
Comparisons of Non-Test Selected and Test-Selected I nventory Managers 

' "~ As described in Chapter II, test scores were seldom or never included 
as part of the process for selecting Medical Technicians or Cartographic 
Technicians. However, Inventory Management Specialists had been required 

• in many instances to take a written test (usually the Federal Service 
Entrance Examination), for entry^ into a career development training program 
at the GS-5 level, leading to the journeymaii level tested in this study. 

In the Personal History Questionnaire, inventory managers were asked to ^ 
respond .to the question: ^ 

••what kind of test wpre you required to take when 

you were hired or reclassified as an "inventory 

manager?^^ 

1 . None 

2. Federal Service Entrance Examination 

3. Other 



expressed in percent of 


ethnic group 


sample were as 


follows: 




No 




Other 




test 


FSEE 


test 


Black 


50 

• 


40 


10 


Mexican-American 


64 


20 


16 


Caucasian 


41 


. 53 


6 



ERIC 
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Table X-7 shows means and standard deviations of selected predictor 

and criterion scores by ethnic group for those who were not tested, those 

who were screened on the FSEE, and those who were screened on some other 

\ - — 

test prior to entry into an inventory management job. It will be noted 

that without exception those who werf screened on the FSEE received 

higher scores on the average on every measure than those who were not 

required to take this test prior to entry. In general, those required 

to take some other test also scored higher on the average than those not 

screened by any test. Differences in supervisors' ratings for those who 

had taken a test as a pre-selection requirement are not as striking as 

those on aptitude, job knowledge, and work sample measures. However, 

supervisors' ratings are higher to some degree in every instance as well. 

Background Variables as Differential Predictors Across Ethnic Groups 

• Cartographic Technicians 

An exploratory analysis of data from Cartographic Technicians was 

undertaken .in- an effort to determine whether certain background vari- 
ables might aff-ect the level of accuracy of prediction across ethnic 

groups. That is, was it possible that some aptitude tests might have 
varying degrees of predictive accuracy for individuals from different 
backgrounds, '^o reduce the number of available background variables, 
a factor analysis of the Personal History Questionnaire was performed. 
The first five factors were selected and rotated using the variance 
procedure. 

The fi-ve factors were called: (1) length- of time as cartographer, 
but not 1371 series; (2) length of . time in lower grades; (3) lengfelb of 
time in 1371 series; (4) socioeconomic standing of parents; and (5) 
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educational and training levi^l. The Cartographic Technicians were then sub- 
divided according to their scores on each of these factors through the use 
of the moderated regression technique (Rock, et al., 1968). Of particular 
interest was the possibility of finding differential prediction for members 
of phe two extreme groups on each factor. For example, different kinds of 
tests might predict more 'accurately for individuals from a high socioeconomic 
background than for thoi,e from a low socioeconomic background. 

The possibility of differential prediction was evaluated for five tests 
against two criterion measures. The tests selected were Necessary Arithmetic 
Operatidhs,' Surface Development, Hidden Figures, Extended Range Vocabulary, 
and Map Planning. The two criterion measures were the "pull-up" task from 
the Work Sample end the Supervisors* Overall Rating. 

Only one of the five factors led to groupings of individuals who 
appeared to be characterized by differential predictive accuracy. That is, 
the two extreme groups on the fifth factor, educational and training level, 
differed with respect to accuracy of prediction on both criterion measures. 
Table X-8 presents the multiple correlations and zero-order validity coeffi- 
cients against both criterion measures for the "low education and training," 
"high education and training," and "total" groups. For both criterion 
measures, the multiple correlation is higher for the "high" groups than for 
the "low" and/or "total" group. Also, as might be expected, the "high 
education and training" group had higher mean scores on all the predictors. 
It was somewhat surprising, however, that the best predictor for the "high 
education and training" group for both criterion measures was the Surtace 
Development Test, while the least accurate fredictor was the Extended Range 
Vocabulary Test. 
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Little if anything of a'conclusive nature can be drawn from this 

analysas, howeve^r, since 'the samples we.re too small for replication. 

•^It does suggest,"^ however, that background factors, which do in fact 

dut across facial groups ttfight also have .an impact on differential 

predictiv^,accura*cy . ' 

Comparison of >Tasks Performed by Ethnic Group Members 
' t • k - 

/\ A d^t^iled task fist was included as part of the Personarl History 
^%filiestiQnnaire- for each of th^ three occupations. Response patterns 

» • ^ A * - 

were^ analyzed by ethnic ^group tc determine whether, for a given occupa- 
tion, the different ethnic groupfe were performing essentially the same 
Ntask^. y ^ • ^ 

Job^ assignments^ of Medical Technicians x^ere examined by GS-level 
to find out what'.ltheir tasks were and how oftei\ they were performed. 
Chi-square values y^re computed to determine whether response patterns 

to the task list^ showSd significant differences within grade leve 

' T ^' ' J 

between ethnic groups.^. No major differences wer^ found. 

Job assigiimentSi*o4 Cartographic Technicians were examined by 
installat|^ by etlir^ic group. Chi-square values weje computed. No 
major differences were disclosed among ethnic groups. A total of 24 
out of 130 tasks- showed a jiif ference in at least one installation. 
However, no* one task was significantiy. different at all installations. 
Six tasks were different at two ^stallatlons. This is fewer than 
would bj^ expected by chance. • . ^ " ^ 

Inventory Management Speciali-sts job assignments were first 

examined by ethnic group. There were few di^^erei\ces between Blacks , 

and Caucasians, but for about 40 items, tshe Lasl^s performed by Mexican- 

t ^ 



Americans appeared to differ from the job activities of the other two 
groups. The data were further analyzed by comparing the responses of 
San Antonio Caucasians (who worked with the Mexican-Americans there) 
with those of all Caucasians. Table X-9 shows a sselection of tasks 
from these analyses. As may be seen, significant differences between, 
the two Caucasian groups are indicated, but there were no significant 
differences between the Mexican-Americans and the Caucasians working 
at the same installation in San Antonio. \ 

It may be concluded that no significant differences in the patterns 
of tasks performed, by members of respective ethnic groups were found to 
exist in any of the occupations studied. 

* 

Summary and Conclusions 
A number of different comparisons of background and task variables 
were made for the three occupations studied, to examine possible ethnic 
group differences which might affect other results obtained. The find- 
ings are summarized as follows: 

1. Differences in composition of ethnic satsples on selected back- 
ground variables wjare negligible for all three occupations. 

2. Distributions of salary grades by ethnic group by occupation 
were not significantly different with one exception, vhere a few more 
Inventory Management Specialists from one ethnic group were added at 
the lowest salary grade level tested in order to increase the ethnic 

sample. * . - 

I 

♦ • 

3. When relationships of selected background variables to aptitude 
and job performance measures were examined, Present salary grade was 
'significantly related to measures of job performance but only in a few 
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instances to aptdtude test. scores for .Blacks and Caucasians in all three 
occupations. This is also true for Mexican-American Cartographic Techni- 
cians. However, for Mexican-American. Inventory Management Specialists, 
significant relationships were found of Present salary grade ^to both 
aptitude and performance' measures. Among other background variables. 
Age tended to correlate positively with vocabulary measures, but ' negatively 

e 

with speeded tests in general. \ 

4. Inventory*-Management Specia'.ists screened -on the FSEE prior to 
job entry received higher average sci\C'i=^ on every measure than those who 
were not tested as a pre-selection r.?quireme;it'. ♦ 

5. Exploratory analyses of Cartogri^phic IVchnician data were, 
generally inconclusive as to the moderating affect? of selected back- 

* ground variables on level of predictive accur.*^-> r^'^ross ethnic groups. 

6. For a given occupation, no major dirf^'^swc.-.*! were found in the 

» 

patterns of tasks performed by members of respectiv* t^trnic groups. 
Apparent ^differences betwe^i Mexican-American ln\ ruifory i anagers and 
members of the othet two ^ethnic groups disappeared wl'en 1 lucasians * - 
working at Sart' Antonio were compared with their Mexican-Auerican / 
counterparts. The task differences were by instalialion 4 ^, Cher than 
by ethnia group/ 
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Table X-1 

Distributions of Selected Background Variables 
Expressed as Percent of Total Sample, by Ethnic Group 
Medical Technicians 

Percent 







Black 


Caucasian 


oex 


na j.e 


HO 


ill 




remaxe 




J J 


Age 


60 + 


2 


2 ' 




50 - 59 


8 


19 




40 - 49 


29 


31 


« 


30 - 39 


43 


22 • 




20-29 


18 - 


25 




Less unan 


n 
u 


1 

X 


Education 


Advanced study 


5 


2 




College degree 


8 


7 




College, more than 2 years 


21 


18 




College, 2 year terminal 


7 


5 • 




College, less than 2 years 


' 32 


31 




High school graduate 


on 






Some high school 


4 


4 


n ^ 


8th grade or less ^ 


- 0 


1 


Source of 
training as 
Medical Technician 


Accredil^ed school 
• Military service 
Government hospital 


40 
17 
23 


31 

28 ♦ 
11 




Civilian hospital 


7 


13 




Civilian laboratory 


5 . 






Other 


6 


10 


Length of 
training as 
Medical Technician 


24 months or lojiger ^ 
18 - 23 months 
12 - 17 months 


16 
11 
49 


li 

6 
52 




Le^ss than 12 months 


23 


31 



267 



» ■ -syl- 
lable X-1 (Continued) 

Percent 









Black 


Caucasian 


Total years 


« 

20 or- more 




8 


25 


of experience 
as a » - 
Medical Technician - 


16 - 19 ^ 
12-15 




14 
21 


12 
16 




8 .- 11 




21 


16 




4-7 




. 18 


17 




2-3 




5 






Less than 2 




10 


8 


Total years 


20 or more 




5' ' 


14^ 


in Civil Service 
as a 

Medical Technician 


16 -19 
12 - 15 


- 


. . 9 
16 


9 
12 




8-11 




16 






4-7 




28 


20 




.2-3 


r 


14 


14" 




Less than 2 




12 


16 


Length of time 


8 years ar longer 




48 


38 


employed at 

present 

installation 


4-7 yearc 
1-3 years 




23 
14 


17 

*,23 




6-11 months 




9 


7 




Less than 6 months 




5 


14 


Present 


8 01* higher 




4 


5 


salary' grade 
(GS) level. 


7 
6 


* 


21 
36 


20 
41 




5 




27 


24 




4 or lower 




12 


10 



I 



/ 
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Table X^2 

Distributions of Selected Background Variables 
Expressed as Percent of Total Sample, by Ethnic Group 
Cartographic Technicians 

Percent 
Mexican- 



ERIC 





< 




^ Black 


American 


Caucasian 


Sex 


Male 




62 


79 


34 , ^ 


\ 
\ 

\ 


. Female 




38 


21 


66 


\ 

Age 


. ' . 60 + 

50 .- 59 




■ / ' 

6 


0 
9 


- 2 
13 • 




40-49 




39 


27 


23 ' 




■ % 30 - 39 




36 


62. 


26 ' 




.20 - 29 


V 


18 


2 


36 



Education 



1 or more y6ars graduate 

r 4 years\ college 
1 or 2 years college 
Tech or Voc institute 
11th or 12th grade 
9th or 10th graVie 
8th grade or less 



0 

20 

41 ^ 

15 

24 
0 
0' 



0 

1. 

27 

■13 
56 
2 
1 



1 

? 
25 
18 
50 
2 
0 



Amount of 


\ 

Quite a lot ^ 


7 


13 


10 


training before 

entering 

05-1371 


A moderate amount 
None, or hardly any 


18 . 
■ 75 


41 
46 


12 
77 


Source of 


High school 


4 


13 


12 


formal training 
as "Car to Tech 


Junior or 4-year colljege 


A 


1 


4 


before entering 


Tech or Voc institute 


9 


9 


2 


05-1371 


In the military 


6 


6 


• 5 




Civilian government 
mapping organization 


53 


' 38 


48 




Commercial mapping 

organization 
* 


0 


• 25 


1 




Other location 


1 


3 


0 




No priqx formal training 


23 


5 


28\ 










\ 










> 



\ 
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Table X-2 (Continued) 

Percent 



Mexican- 
Black American Caucasian - 



Source of 


At a Tech or Voc institute ' - 


1 


2 


0 


on-the-job 
training as 


In the military 


3 


. 3 


4 


Carto Tech 


At a commercial map-making 








before entering 


organization 


0 


24 


2 


1371 series 


At a governmental map- 






56 




making organization 


70 


56 




Other 


1 


2 


1 




No prior 'on-the-job 










training 


25 


13 


37 


Length of 


More than 8 months 


35 


. 32 


32 


training ^ 


7-8 months 


10 


1 


■ 4 


after entering 






16 


1371 series 


5-6 months 


13 


18 




3-4 months 


21 


23 


22 




1-2 months 


13 


13 


11 


•* 


Less than 1 month 


8 


• 13 ' 


15 


Total years • 


20 or more 


4 


3 


4 


of experience 


16 - 19 


20 


7 


15 


as Carto Tech 








in 1371 series 


12 - 15 


21 


27 


13 




8-11 


13 


34 


14 




• 4 - 7 


26 


27 


37 




2-3 


14 


1 


14 




Less than 2 


1 


0 


3 


Total years 


20- or more 


1 


7 


4 


of Experience / 


16-19 


10 


8 ■ 


6 


as Carto Tech / 




16 




outside 


12 - 15 


6 


6 


1371 series / 


8-11 


12 


12 


12 




4-7 


14 


27 


23 




2-3 


13 


12 


11 




Less than 2 


44 


17 


38 

1 
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Table X-2 


(Continued) 


Percent 




* 




, Black 


^ Mexican- 
American 


Caucasian 


Length of^ time 
employed at 
present 
installation 

! 


8 years\or longer 
4-7 yeaVx^ 
1-3 years ^ 


69 
• 23 

r 

5 


90 
10 

n 
u 


57 
35 

\j 




6-11 months 


3 


n 
u 


1 

X 


* 


Less than 6 mdnths 


0 


0 


X 


Present 
salary grade 
(GS) level « 


12 \ 
11 \ 
10 


1 

5 

\ ^ 


0 

0 ' 


.0 
8 
0 




9 


\ 


83 


55 




8 


\/ 10 


0 


8 




7 * 


.. \ 32 


17 


23 




6 


\ ° 


- 0 


0 




5 


\o 


0 


5 



ERIC 



-379- 

. Table X-3 / 
Distributions of Selected Background Variables.? 
Expressed as Percent of Total Sample, by Ethnic GroOp 



Inventory Management Specialists 



Percent 

Mexicfan- . 
Black American Caucasian 



Sex 



Male 

Female 

No response 



37 
58 
5 



52 
38 
10 



Age 


60 + 


5 




6 




50 - 59 


25 




33 




40 - 49 


42 




29 




30 - 39 


18 


i 31 


13 




• .2Cr - 29 

J 

No response 


4 
6 


/ 3 
i 5 


9 

10 



Education 



Graduate school 

3 or 4 years college 

1 or 2 years college, 

1 or 2 years tech or 
business institute 

11th - 12th grade or 
GED diploma 

9th - 10th grade 

8th grade or less 

No response 



3 
20 
28 

17 ■ 

27 
0 
0 
5 



3 
15 
26 



41 
3 
1 
3 



2 
22 
9 

19 

34 
4 

0 

10 



Amount of 
training 
before eYitering 
GS-2010 



Quite a lot 
A moderafc' noUnt 
None, or hi. Jly any 
No response 



13 
10 
62 
15 



5 
16 
■ 73 
5 



6 
8 
73 

13 



ERIC 2.72 
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Table X-3 (Continued) 



Black 



Percent 

Mexican- 
American 



Caucasian 



Source' of 
formal training 
in -inventory 
management 
before entering 
2010 series 



High school 

Junior college 

Pour-year college 

Tech or business institute 

In the military 

0 

Civilian governmental 
activity 

Non-governmental 
commercial organization 

Other, location 

No prior formal training 

No response 



8 
3 
2 
6 
6 

23 

0 
1 
37 
15 



11 
3 
7 
7 
8 

18 

1 
0 

40 
5 



1 
3 

3 
6 

21 

1 
0 
45 
14 



Source of 
on-the- j ob 
training 
before entering 
2010 series 


Work-study program 

In the military 

Civilian gpvernmental 
activity 


0 
4 

44 


1 
7 

46 ' 


0 
6 

34 




Non-governmental 
commercial organization 


1 


5 


3 


\ 


Other 


3 


3 


2 


\ 


No prior on-the-job 
training ^ 


36 


34 


43 




No response 


12 


4 


12 ■ 


Length of 
training 
after entering 
2010 .series 


More than 8 months 
7 or 8 months 
5 or 6 months 


27 
4 

11 


15 
0 
9 


» 

18 
2 
8 




3 or ''4 months 


19 


12 


16 




1 or 2 months 


^ .9 


23 


20 




Less than 1 month < 


18 


34 


22 _ 




No response 


12 


7 


13 
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Table X-3 (Continued) 

Percent 
Mexican- 



i 

1 




4 

/ 


Black 


American 


.Caucasian 


Total years of 


/O 


or more 


A 


0 

« 


4 


experience 'as 


16 


- 19 


5 


3 ' 


6 


Inventory 


18 


15 




Management 


1 9 


— 1 

— ±D 


10 


Specialist 


8 


- 11 


23 ' 


26 


19 . 


in 2010 series 






38 

\ 


4 


- 7 


^IC 




2 


- 3 




1 9 


ft 




Les 


s than ^ 




5 


5 




No 




7 


. 5 


10 


Total years of 


20 


or more 


A 
4 


■ 


6 


OVT^ ^T"l ^TlO ^ 1 Tl 


16 


- 19 




/, 


9 


inventory 






4 


management 


12 


- 15 


6 


7 




outside ' 


8 


- 11 


10 


7 


4 


2010 series 








• 


4 


- 7 


xx 


1ft 


• 12 




2 


- 3 


7 


4 


3 




Les 


5S than 2 


35 


35 


45 




No 


response 


24 


24 


■ 23 




Length of. time 


8 years or longer 


bo 






employed at 


4 • 


- 7 years 


xo 


q 


36 


present 






3 1 


8 


installation 


1 - 


- 3 years | 


6 








6 • 


- 11 months 


2 


0 ' 


1 




Less than 6 months 


0 


0 


0 




No 


response 


5 


3 


10' 


Present 


11 




21 ' 


8 


20 


salary grade 


9 




67 


68 


66 


(GS) level 


7 




6 


19 


4 




No 


response 


6 


5 


10 
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Table X-4 

Correlations Between Present Salary Grade and 
Selected Predictor and Criterion Variables 
Medical Technicians =. 



1 



Aptitude Tests 

Subtraction & Multiplication 

Vocabulary 

Hidden Figures 

Necessary Arithmetic Operations 

Pin-Dexterity 

Number Comparison • 

Gestalt Completion 

Picture-Niimber 

Paper Folding 

Criterion' Variables 
Learning Ability Rating ^^^^ [ 
Job Knowledge Rating 
Supervisors* Overall Rating 
Job Knowledge Test Score 



Black- 


Caucasian 


N=168 


, N=297 


.15* 


-.01 


.02- . 


-.09- 


.13 


-.02 


.17* 


-.15** 




-.07 


.04 


-.13* 


.03 


-.13* 


.15* 


-.05 


.00 


.-.12* 


.37** 


.24** 


.50** 


.44** 


.45** ■ 


.29** 


.48** ■ 


.21** 



* significant at .05 level 
** significant at .0.1 level 



I. O 

ERIC 
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Table X-5 

Correlations Between Present Salary Grade and 
Selected Predictor and Criterion Variables 
Cartographic Technicians (TOPOCOM) 



Aptimde Tests 
Hidden Figures 


Black 
N-101 


Mexican- 
American 
N=101 


Caucasian 
N=240 


.13 


-.15 


.10 


Vocabulary 


24* 


-.08 


.08 


CS Arithmetic 


;04 


-.06 


-.04 


Man Plprminp 


.02 


-.14 


-.06 


Surface Development 


.05 


-.14 


-.02 


Necessary Arithmetic Operations 


.09 


-.13 


-.06 


Criterion Variables 








Work Sample Composite 


.21* 


.15 


.08 


Learning Ability Rating 


.42** 




, 33** 


Technical Knowledge Rating 


. 54** 


.20* 


.48** 


Supervisors* Overall Rating 


. 50** 


.19 


.40** 


* significant at .05 level 
** significant at .01 level 




/ 





\ 



ERIC 
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Table X-6 

Correlations Between Present Salary Grade and 
Selected Predictor and Criteri^^n Variables 
Inventory Management Specialists 

Mexican- 





DiacK 




Caucasian 


Aptitude Tests 


Nj=112 




M~ 1 Q A 
IN- J.?H 


Number. Comparison 


-.09 


1 c 

• J. J 




Hiddc Figures 


-.16 


.23* 


n 7 


Vocabulary 


-.08 


.23* 


-.10 


Nonsense Syllogisms 


.06 


.24* 


-.11 


Subtraction & Multiplication 


.00 


.27* 


.02 


Necessary Arithmetic Operations 


-.17 


.39** 


-;06 


FSEE ' 


.02 


' .38** 


-.13 


Criterion Variables 








Work Sample Overall Performance 


.22* 


.40** 


-.08 


Learning Ability Rating 


.03 


.24* 


.10 


Technical Knowledge Rating 


.06 


.3b** 


.22* 


Supervisors' Overall Rating 


-.05 


.17 


.18* 



* signiEican-. at .05 level 
** significant at .01 level 



ERIC 



£77 



-391r 

Table X-7 , " ^ - ^ 

Means and Standard Deviations of Selected Predictor- 
and Criterion Scores — Those Not Tested vs. Those Tested 
Before. -Wiring or Reclass-if ication 
Inventory Management Specialists 



Black 
N=50 N=40 

Nq 
Teat 



N=10 



FSEE 



Aptitude Tests 

Number 
Comparison 

Hidden 
Figures 

Vocabulary 

Nonsense 
Syllogisms 

Subtraction & 
Multiplication 

Necessary 

Arithmetic 

Operatipns 

FSEE 

Work Sample 



M 
'Sp 

sd' 

M 
SD 

M 
SD 

•M 
SD 

M 
SD 

1 



38. 5 \ 42\8 
11. 4-; iO." 

4.5 
'4.3 

20.2 
5.8 

5.4. 
6.4 • 

,62. 7 ■■ 
'l9 . 4 

9.8 

.f / 52.0 
.42.7 




40.6 
10.4 



Mexican -American 



No 
Test 



. 41.3 
10.3 

6.2 
' 4.6 

20.0 
6.0 

#.-5.'l 
5.0 

65.7 
20.3 ' 

12. :y- 

5.0 , 

■38.8 
9.9 



Caucasian 



N=14 


N=li 


N=73 


N=95 


N=12 




Other 


NO 




utner 


FSE-E 


Test 


Test 


FSEE 


Test 


46.7 


43.6 ■ 


37.8 


43.1 


45.0 


9.5 


9.9 


13.0 


9.7 


11.2 


9.5 


7.1 


. 5.5 . 


9.2 


8.8 


5.0 


5.2 


5.2 


5.6 


4.5 


24'. 9 


11.1 


22.9 


28.5 


24.8 


4.2 


6.4 


6.2 


4.3 


6.2 


10.6 


6.4 


5.6 


11.7 


10.3 


'6.6 


7.0 


5.9 


8.5 


9.1- 


7^7.6 


84.7 


71.8 


78.8 


88.9 - 


12.6 

— » 


19.9 


21.8 


20.. 9 


18.6 


17.5 


15.0 


.,12.5 


'18.0 


17.5 


4.6 


. 5.5 


■■ 5.2 


3.9 


4.9 


52.9 


51.0 


40.5 


60.8 


' 4'9 . 2 


11.1 


10.0 


12.3 


10.6 


10.1 



Scorer's Rating 
of Overall 
Performance 


'M 
SD 


6.0 
3.0 ' 


7.7 

' 3.6 


V 

■ 7.1 
2.6 


7.7 
3.8 

\ 


11:8 
3.8 


10.4 
. 3.7 ■ 


7.3i . 
. '4.1 


9.7 
4.6 


10.1 
3.6 


Rating Scala.s 






















Learning 
Ability 


M 
SD 


5.9 
i.9 


6.4 
1.9 


6.3 
1.3 


5.9/ 
1.7 


^6.8 
1.3 


6.0 
2.2 


5.7 
1.8 


. 6.5 
1.8 


5.8 
1.8 


Technical 
Knowledge 


M 
SD 


5.6 
2.1 


6.2 
1.7 


5.9 
1.5 


6.2 
1.6 


6.8 
1.3 


6.2 
2.0 


5.9 
1.8 


6.5 
1.6 


6.0 
1.9 


Overall 


M 
SD 


6.2 
1.8 


6.4 
1.7 


6.4 
1.3 


6.3 
1.8 


7.3 
i.2 


6.0 
1.7 


6.0 
1.7 


6.6 
] .5 


5.7 
2.0 



/' 
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Chapter XI , 

An Alternative Approach to Culture Fairness 

^ when Tests are Used in Selection 

^^Recently a number of definitions of "test bias" have been proposed* 

The one definition which has beeo most often applied is the Cleary (1968) 

definition^of "test bias." Simply stated, Cleary defines a test as being 

"culture-fair" for populations A and B when the regression equation based 

on population A neither systematically over- nor- under-predicts level of 

performance for members of population B. Thorndike (1971) has proposed 

an alternative definition which can lead to entirely opposite conclusions 

with respect to whether or not a test is "culture~f axr in a prediction 

situation* 

Thorndike suggests that a test may be judged "culture-fair" if the 
overlap on ^the criterion scores between groups A and B is essentially 
equivalent to their overlap on the predidtors. 'Assuming normally dis- 
tributed criterion and ' predictor scores within each group, and equivalent 
standard deviations across groups, the measurement of overlap reduces to 
comparing the differences between group means scaled by their standard 
deviations on the. criterion and ofi th^ predictor. For example, if the 
means for group B are one standard deviation apart on the criterion^ they 

should also be one standard deviation apart on the predictor; If the two 

f • 

grouf^s'^means are only one-half standard devia^/ion apart op the criterion 

but a full standard deviation apart on the predictor, the test would be 
judged unfair in the sense that for any one specified criterion cutting 
score, the members of each group would not have the same opportunity fdr 
being selected as would be represented by the proportion of their group 
" • ' -397- 
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well as the reliability of such criteria. Past research suggests that 
more often than not, criteria such as ratings tend to have lower reli- 
abilities (i.e., inter-rater reliabilities) than the objective tests 
used to predict them (Conklin, 1923; Stead ancf Shartle, 1940; ^Ghiselli 
and BrDwn, 1955).. Thus, before any conclusions can be drawn with 
respect to culture-fair test usage based on the Thorndike approach, it 
would seem reasonable to correct the criterion scores for unreliability. 

The fact that the subjective nature of ratings almost rules out 
the possibility of "color-blindness" can lead to even more serious 
proVfems." The possibility of bias^in ratings is cocumented in other 
reports forthcoming from the present study. Of course, the possibility 
of bias in a subjective criterion tends to reduce the usefulness, if 
not appropriateness, of the Thorndike criterion, or for. that matter, any 
definition regarding ^^culture-f air" usage of tegts. 

Results of a modified Thorndike analysis of culture-fair usage of 
tests against (1) a subjective rati«g criterion corrected for unreli- 
ability, (2) an objective x>?ork sample criterion corrected for estimated 
unreliability, and (3) a criterion based on an objective test purporting 
to measure job knowledge are presented here. The analyses were performed 
for the ethnic samples in the three occupations in the present study: 
Black and Caucasian^Medical Technicians; Black, Mexican-American, and 
Caucasian Cartographic Technicians; and Black-. Mexican-American, and 

ft 

# 

Caucasian Inventory Management Specialists. 

The correction for unreliability in- the subjective ratings was made 
in an effort .to see if the "true" mean difference in criterion rating . 
means more closely approximated the difference in predictor scora means. 

V, 

'/ 1 

^ 2S2 . 
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The hypothetical "true" mean difference may be defined as that difference 
J>etween minority and majority group means which one would expect to find 
ir the criterion rating was as' reil^iable as the predictor test. The 
assumption here, of course, is^'that biaa-^n the ratings is at a minimum. 
However, additional insight into the possibility of bias in the ratings 
may be inferred from the comparison of r^elative differences between group 
means on the subjective rating criterion ^and the group mean differences 
which were found when an objective job knowledge test and/or work saAple 
was used as^a criterion. Admittedly, a paper-and-pencil test has its 
drawbacks due to its somewhat theoretical nature. ^ It would be fairer to 
point out here that a possible drawback of a job knowledge test as a 
criterion is that some of the same skills- which play a role in the pre- 
dictor also affect the criterion scores (e.g., "test taking" skills, if 
in fact these exist other than in a hypothetical sense). However, it 
does have the following advantages: (1) in general, i^:s reliability 
approximates that of th^ test used to predict it, (2) it is "color-blind," 
and (3) in this particular situation, it was deemed to be an appropriate 
measure of job knowledge by supervisory staff members who participated 
in developing the test. 

The corrections for unreliability of ratings were originally suggested 
by Angoff (in a personal .communication) and were estimated in the following 

manner. Two critical riatios were^formed, one for the criterion means, . 

^^^^^ ^ 

CR^ , and one for the^ predictor Tpeans, CR . 
yy ' I XX 

CR' = - Y (1) 

yy sif ^ ■ 
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\^ere: Y = mean score on the criterion for the Caucasian group 

Y = mean score on the criterion for the Black grouo 
B 

<* 

and: 




where: . = variance of the Caucasian criterion scores 
= variance of the Black criterion scores 

B 

r - reliability of the criterion rating 

yy 

r - reliability of the predictor test 

XX 



while CR is simply: X - X 

XX C B 



a 

X 

Equation (1) above simply expresses the expected difference between the 
group means on the criterion (in terras of the standard deviations of their 
true scores) if the criterion ratings had the same reliability as the pre- 
dictor. The prirae in the ratio CR"^^ indicates that this correction for 
unreliability has been made. The reliabilities, based on inter-rater 
agreement, were .46 for the Black Medical Technicians and .66 for the 
Caucasian Medical Technicians. • The average reliability was then used in 
equation (1). lu short, if the predi'^tor and the criterion were equally 

reliable, would CR" equal CR ? If the answer is yes, then one could 
' yy ^ XX 

conclude by this modified Thorndike definition that there was no unfairness 
in selection against either group and that the "unfairness" found before 
the correction was simply due to differential reliability between predictor 

and criterion. However, if CR is still significantly larger than CR^ , 

* XX , y J 
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one may conclude that the test is biased against the Black group. The 
assumption here 'is that the Caucasian means ^,e higher than the Black 
means and that there is little or no racial bias in the rating pro- 
cedure. 

. ? 

Results* and Discussion t . 

<• 

Medical Technicians 

Table XI-1 presents the critical ratios for each of the predictor 
tests against each of two criteria for Medical Technicians. In all 

cases, the critical ratios, CR^ and CR , are both positive, 

j yy XX 

indicating that the Caucasian, means were higher on both the predictor 

i 

and the criterion. For example, in the case of the Subtraction and 
Multiplication Test predicting' the Job Knowledge Test, the CR^^ is 
equal to .44, indicating that the^ Caucasian group wa^ approximately 
one-half standard deviation above the Black group on the criterion, 
while the CR ratio indicates that the Caucasian group was also about 

XX 

one-half standard deviation above the Black group on the Subtraction 

and Multiplication ^est. When tests were used for criteria, no 

correction for unreliability was made since r^ then approximated r . 

XX yy 

Using the Thorndike definition, one would conclude that there is 

little or no unfairness against the Black group when the Job Knowledge 

Test is used as the criterion and the Subtraction and Multiplication 

Test is the predictor. Similar^ findirgs apply to the remaining tests, 

with the exception of Vocabulary and Necessary Arithmetic Operations. 

Out of the eight possible predictors, only these two tests show a 

significant bias against the Black group when the Job Knowledge Test 

is the criterion (i.e., CR is significantly less than CR ) , vrhere 
\ . yy XX 
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Significance is arbitrarily defined as a critical ratio difference of at / 
least one-quarter standard deviation. 

i \ 

An inspection of the critical ratios for jthe predictor' tests against 

' ) ( ' ' 1 ^ Ji 

the Overall rating would seem to lead to the opposite conclusion, ' Closfer 

inspection of the Black and Caucasian validity coefficients for each of 

I 

the tests suggests that they are almost all near zero and, as a result, 
the" application of any definition of culture If^rness would be meaning- 
less. Either the tests are inappropriate and/or the criterion ratings 

f 

include a reliable but large proportion of invalid variance. Results 

I 

from earlier studies concerning supervisors? ratings among Medical Techni- 
cians lend some support to a hypothesis of jbias in the supervisors* ratings 
The finding that certain ethnic combinations of rater and ratee lead to 
systematic but not necessarily valid sources of variance in the overall 
ratings may help to explain why ubjectivej ("color-blind") predictors 
have little variance in common with such a criterion. 
Cartographic Technicians | 

Table XI-2 shows similar data for Black, Mexican-American, and Cauca- 
sian Cartographic Technicians. More speiif ically , Table XI-2 presents the 

critical ratios for Caucasians and.Blaclcs and Caucasians and Mexican- 

f 

Americans when the Supervisors* Overall^ Rating was used as the criterion. 
Of the four predictors (Hidden Figures,! Map Planning, Surface Development, 
and Necessary Arithmetic Operations) having similar non-zero validity co- 
efficients, no one test was consistently unbiased against all groups. 
Hidden Figures and Map Planning could be used as **culture-f air" selection 
instruments '(against the rating criterion) if there was only a Caucasian 
and Black applicant pool. Howevev, if the applicant pool also included 
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Mexican-Americans, these two tests would, no longer be "culture-fair" instru- 
ments, since they would tend to discriminate against the ^Mexican-Americans 
and in favor of the Caucasians. 

It should be noted here that a test may discriminate in favor of the 
majority group as well as cert^l^minority groups, but not necessarily all 
minority groups. However, before any conclusions can be drara from, the 
above analysis concerning the culture fairness of tests against supervisory 
ratings, additional evidence concerning the vklidity of the supervisory 
ratings should be presented. ?The* Cartographic Technicians' study 'was 
especially appropriate for such an analysis since three types of criteria 
were available for comparison. Two of these criteria, a paper-and-pencil 
Job Knowledge Test and a comp'osite Work Sample, were objective "color-blind" 
measures, and the third was the Supervisors' Rating. 

\ As pointed out before, a paper-and-pencil job knowledge test may be - 
unsuitable as a criterion for identifying unfair test selection procedures 
sinc^ it may be argued that the theoretical responses to an "artificial" 
job situation may not reflect the skills necessary for the successful 
performance of, the actual job tasks. Figure XI-1 presents; a paradigm * 
indicating the relationships between the three criterion measures. 
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Inspection of Figure XI-1 suggests that the Job Knowledge Test is 
more highly related to the actual tasks performed on the job than is the 
subjective Supervisors' Rating.' Since ratings are not "color-blind," it. 
is possible that they may include nonr^levant variance due to. racial 
bias.^ It is particularly interesting to note that two of the three 
correlations between the Supervisors' Overall Rating and the Work Sample 
eomposite "are quite low, while the correlations, between the two objective 
measures, one "theoretical" and one operational, are all close to .50. 
It would seem that the "theoretical" job knowledge measure has much in 
common with the objective work sample, yet neither have much in common ^ 
with the subjective^ ratings. 

Table XI-3 presents the critical ratios^for the various group com- 
parisons when the Work Sample Composite is used , as a criterion. Of the! 
six tests which have similar non-zero correlations withrtlie Work Sample 
criterion — Hidden Figures, Card Rotations , Map Planning, Surface 
Development, Maze Tracing Speed, and Necessary Arithmetic Operations — 
Card Rotations, Map Planning, and Surface^ Development^ appear -to be 
relatively culture-f ar.r regardless of the groups to be compared. Map 
Planning,, in fact, h'as a slight tendency to favor the Blacks and 
Mexican-Americans when these minority groups are compared with the Cauca- 
sian majority. It is also interesting to note that; besid^ b2i}tjg 
relatively culture-fair, these tests have relatively high f aceNjirar^dity 
for this particular job. 

Table XI-4 presents the critical ratios for the various group com- 
parisons when the^Job Knbwledge Test is used as a criterion. Hidden ^ 
Figures, Vocabulary, Card Rotations, Map Planning, Surface Development, 



k87 



-406- 

Maze Tracing Speed, Extended Range Vocabulary, and Necessary Arithmetic 

^^rations all had non-zero validity coefficients across all groups. In 

general, when the Caucasians are compared with the Blacks, only one test 

out of eight appears to be biased against selecting the "appropriate 

proportion of^Blacks, Kiid that test is Necessary Arithmetic Operations. 

In f.act, five ouc of the eight tests are slightly biased < ^^^yy^ 

'in favor of the Blacks. These tests are Hidden Figures, Vocabulary, 

< • 

Card .Rotations, Map Planning, and Extended Range Vocabulary. When Cauca7 
sians. are compa:.^- with ^lescican-Americans with respect to these eight 
predictors, the bias, if any, is always in favor of the Mexican- Americans. 
As in the case of the Medical Technicians, when tests were evaluated 

t 

against an objiictive criterion, little or no^predictive biaS^as found. 
Inventory Management Specialists 

Table XI-5 presents the critical ratios for the predictor testes 
against Supervisors* Overall Rating for Inventory Management Specialists. 
Only two predictors. Subtraction and Multiplication and Necessary Arith-^ 
metic Operations', out of ten prediptots show similar non-zero relatdon- 
ships with the rating criterion. In both instances, the critical ratios * 



indicat^e that these^^two tests are biased Against both the Black and the 
* Mexican-.\merican minority grou^. However, Tab^e XI-6 shows that whe>» 
the Work Sample overall scores are u^ed as the criterion, six predictors — 
Hidden Figures, Vocabulary, Letter Sets , Extended Range Vocabulary, 
^Necessary Arithmetic Operations, and Inference — all have non-zero validity 
coefficients across all groups. When the Caucasian and, Black Inventory 
Management Specialists are compared, the critical ratios CR^^ and CR^^ 
are approximately equal for five of these six tests, indicating* lite] 

., - • \ . . . 

• , \ • 



or -no predictive bia's^ Necessary Arithmetic 'Operations t^nds to be biased 
agaiiist-the Blacks (CR - CR * t). For the remaining five tests there 
is a slight tendency for the selection proQedure td^be in f.avor of the 
Blacks •(CR > CR ) • However, when the\Caucasian group is compared with 

- ■ . " C . ' . - . ' 

the. Mexican-American group, the "bias" is reversed. That is, with the 
possible exception of the Hidden Figures Test, the remaining five predictors 

■ ■ ■ ■ 

-appear to be biased against the Mexican-Americans. Further inspection of 
the types of prejlictors which lead to the greatest, as well.as the least, 
bias against the Mexican-Americans suggests that possible language^ problems 
Hamper^dMtheir- performance to a 'greater degree on the predictors than on 
the Work Sample criterion.. For example, the two most biased tests against 
the Mexican-Americans were the two vocabulary tests, while non-verbal tests 
such as Number Comparison and Hid'den Figures showed little or no bias._^ 

Conclusions 

With few exceptions, the direction of differences between majority 
and minority group means on the predictors was also reflected in the 
criterion means corrected for unreliability. 

Howeverv when Supervisors* Ratings were used as the criterion, the 

difference between minority and majority group means on this criterion 

f 

were much less pronounced^ than their corresponding difference on, the 
.predictor means. On the surface, this finding suggests that tests were 
biased, in the Thorndike sense, against the minority group when sub- 
jective ratings were used. However, other analyses of the rating data 
ft 

which are reported elsewhere in this report (Chapter VIII) seem to 
indicate a potential ethnic bias in the ratings. The finding of- a 
possible racial contamination of the Supervisors* Ratings was 

c 
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particularly noticeable. for the Medical Technicians. Additional evidence 
against^using subjective ratings as the criterion in a "test bias" study 

Awas also found in the analysis of the Cartographic Technicians data. 

When the Job Knowledge Test and/or the Work Samples werfe used as the 
criterion, much of the evidence for "test bias" against Blacks and Mexican 
Americans disappeared. One exception to this was the Mexican-American 
Inventory Management Specialist^ group. Compared to the Caucasian group, 
t'bey tended to do better, on the average, on the Work Sample than their 
predictor scores would suggest. It is quite possible that a language 
problem did hamper their performance on particular predictor tests used j- 
in this study. The fact .that tfie two verbal predictors showed the great- 
est amount of ".bias" against the Mexican-American group lends additional 
support to a possible language problem. — 
The results from this study suggest that if we accept an objective 

^test or worH sample as being mor^ desirable measures of job performance 
thaii supervisors^ fatn.ngs.,_and carefully select predictors which indeed 
do measure aptitudes and skills relevant to successful_ger£ormance on . 
the job, tests will lead to fair selection procedures for the Black 
iftiriority group. It should, however, be pointed ouu that one might reach 
different conclusions if other than the Thorndike definition were used. . 
Findings with respect to culture-fair selection of Mexican-Americans 
were less clear. 

•Based on' what we have learne3 from the present study, as well as 
from earlier studies, concerning the problems inherent in achieving un- | 
biased supervisors*' ratings, it ma,y well be that the use of an objectivej 
job knowledge test' in lieu of the more subjective supervisors* ratings 
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is /the only reasonable answer^,. This need for a "color-blind" criterion 
wbuid- seem to be particularly germane in those cases where' the {Potential 



bias in the, ratings may have an ethnic ]basis\ 



■ / 



This possibility of -criterion bias raises. Interetsting qtiesttons \ 
with respect to the nature of ,test bias. That is/ all approaches have 

^ ■ ■ *' "■ / ■/ ; \ 

assumed that mean differences in criterion jaeasures .between groups ar^e 
'in fact true differences between the -performance, levels of the group 
members i. Any predictor that does not accurately reflect these^ criterion 
differences is rejected as aiJ'feiased^'-'predictor If, howev.er some or 
all of these criterion differences have been due to bias and mot to 

Z-C" " ■ ■ ' :■/ : : ' ' - ' 

differences in ."true"\ performance, ,the use of differential prediction 
:iias. acted to maintain the bias in selection either, for or against t^e^ ' 
minority group. Future "test .bias" research should settle on a sub- 
jective criterion measure siich/^s ratings only 4,f the ratings can be, 
shown to reflect to a high degree the same rank ordering obtained on ♦ 
"hard'^ criterion measures, ^such as objectively scores job samples, 
which can be considered r^sonably "color-blind." • % 



/ 
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Table XI-1 

Critical Ratios 

Medical .Technicians- 
\ 

Criteria 

Supervisors* Overall Ratdng, 
Black. Caucasian 



Job Knowledge Test 



Black Caucasian 



-Predictor 



CR^ 



CR 



XX 



r 



xy 



CR.. 'CR 

,yy^' XX 



xy 



Subfcfa^^^ &^ 
fiul rip^^^ ion 

iVocaWl^ 

rHiddfen f igur es 

^Necessary . 
Arithmetic 



Nuinber 

iafiisofi 



Gestait 
Cdmpletipn * 

Pittiire'^; 
Number 

..Paper Foidlhg 



.12 . 

.12"- 
.11 

.11 

".12 
.12 

.11 
.11 



.57 .29 



.77 
.47 



.09 
.02 



.84 .17 

.61 ,17 
.33 .05 



.36 -.02.. ' .15 



.65 .08 




-< • " 

Table XI-2 

i 

Correlations and Critical Ratios— " 
^Aptitude Tests with Supervisors^ Overall Rating 
Cartographic Technicians (TOPOCOM) 



Correlations xy 



Critical ratios 





Aptitude Te^ 


Black 


we XX can- 
American 


Caucasian 


Caucasian/ ' 
Black 


Caucasian/. . ■ 
Mexican-American 










I 


CRA, 
yy • 


CR- ,• 

XX 


vv 

\ . ' 


' -CR'~7^ ■ . 
xx' 












y i. ^ ' * 






■-/- ■ ■• 




Hidden Figures 


.21 


.29 


.21 


.24. 


.22 


.07 , 


.3>" ■ 




* yodabulaf 


.19. 




.01 ; ' 


v27 


.12 


.08 


.66 


{* $ 


ObjectrN^ " 


M 


. Ql " 


•■ .02 


.24 


.36 


" .08 


Ti04r^' 


i' - 


Car 3? -Rojt^tions^ 


.16.,. 


.03 . 


s- ,26 


..;27 


.28 


.08 


.34 


J Map^^^-^^^^ 


.24 


.23 


.30 


.26 


.29 


.07 ff 


.35 


y- ' - 


' Surf ace/^ " 
'T)p\7p1 oniTipnt 


.28 


-.22 


' .28 - 


.27 


.49 


i08 


.60 




Maz% Tracing 

Speed;, ^ ^ 


.14 " 


' ■ .15 


.27 


.28 


.53 




v44 


'i 

t " 

4 


' Ext ended. ^Range 
Vp cabu laxy 


.17 


o ■ 

.03 


-.07 


..27 . 


,.14 


.08 


,83 -A 

* 




"Necessary 
Arithraetic 
Operations * / 


.25 


.22 

t 


.19 


.25 

A* 


.82 


.08 


.7.-6 






\ 










i 


^ 9 



\ - 
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Table XI-3 

Correlations and Critical Ratios — 

Aptitude Tests with Work Sample Composite 

•1" * 

'Cartographic Technicians (TOPOCOM) , 





' Correlations 


xy 


.Critical 


ratios _ 




^ Apfcitude Tests 


^ Black 


Mexican"' 
American 


Caucasian 


Caucasian? 
Black < 


Caucasian/ 
Mexi can-Amer i c an 










CR' 

vv 


CR j 

XX 


CR' 

VV 


CR„ 

XX 


Hidden^ Figur e^^ , 


* 

.35 


.43 


,29 


.20 


. 1 
.23 


r ' 
1 

■ -^i 




VocaBulary^ 


..22' - 


-.02 ' 


.22 ' 


.40 


.08 


.46 


.66 / 




:05- ' 


' ' .03 


,11- 


.39 


.36 


-.44. 


-.03 - 


CardffRot: ^ 


,26 ■ 


.43 


.34 . 


.40 


.29 


.47 


.36 . 


« 

Mapr ■Piannihg 


.29 




.24 


.38 


.29 


.44 


.36 


, Surface: 
Deve lopmen t 


.31 


.38' 


.41 ■ 


0 


.50 ■ 


.45 


.61 


Maze fracihg ' 
Speed , 


.30' 


.45 


.34 


.41 


.54 


-.48 


'.46 


Extended 'Range 
Vocabulary 


•< 

.18-' 


.05 


.23 


.4! 


.11 


.47 


.'84 


Necessary 

Arithmetic 

ODeratibhs 


• 

.28 

• 


.3? 


,35 


;38. 


.82 


.44 


■ .•ti> 

.77 



29^ 
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• Table xr-4 
Correlations and Critical Ratios — 
Aptitude Tests with Job Knowledge Test 
Cartographic Technicians (TOPOCOM) 



Aptitude Tests 



Correlations xy 



/ 



Critical ratios 



^ Mexican- ^ 
Black American Caucasian 



Caucasian? 
Black 




'CaUjC^sian/^- " 
xi c aPn-Amer ic an 





- 






CR 

yy 


' CR 

XX 


CR\ 
yy 


CR" 

XX 


-Hi Hrl ^n" -Pn 


40 


.45 




.43 


.23" 


.92 


.42 


Vocabulary. 


.46 


.22 


.39 


.43 


.12 


. .90 


.67 

0 


Object-Ni^ber 


.28 


.io 


a? 


.43 


.36 


.90 


.-.0'3 


iCard^ Rotations - 


.34 . 


.36 


.32 


i43 


.28 


.90 


' . .34 


Map Piaiihing 


.35 


.42 


. .36 


.43 


.29 


.90 


.35 


-Surface 
Development 


.58 


.55. . 


.55 


.43, 


.50 


.89 


.59 


Maze Tracing 
■Speedy: / 


.36 ° 


.46 


.37 


.43 


.53 


.90 


.44 


* * ^ - J- 
Extended Range 

. Vocabulary 


.50 




.39 \ 




.14 


.90 ■ 


.84 


Necessary 
Af itimietic 


.66 


.44 


.54 


.43 


.83 


.90 




dperatiohs 
















* • 




{■ 














« 








» 


.■ * 
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r Table XI-5 - 
Correlations and Critical Ratios — 
Aptl^tude Tests with Supervisors' Overall Rating 
inventory Management Specialists 



Aptitude Tests 



Correlations xy 

Mexican- 
Black American Caucasian 



Critical ratios 

Caucasian/ Caucasian/ 
Black Mexican-American 



CR^ 



yy. 



CR 



XX 



CR^ 



yy 



CR 



XX 



Number 
Comparisoh: 

H idden . Figure s 

Vocabulafy 

ObjectrNumber 

,-Letter.Sets. 

Nonsense 
Syllogisms 

Subtraction & 
Mult ipii cation 

Extended^ Rang^ 
Vocabulary/ 



Necessa(ry 

Arithmetic 

Operations 

Inference 



.29 



.34 



.23 



.10 



.28 



/' .06 



.12 



-.06 



.25 



.131 



.25 



.20 



-.21 



.30 


° .25 


.10. 


.07 


.43 


.04 


.'12 




.14 


• ,.04- 


.12 


.06 


.50 - 


-..06 


.71 ■. - - 




.04 


iOO / 


.07 


.06 


.07 


-.06 


..50 . 




.32 


/ 

/ 


.21 


.06 


.49" 


.08 


.28 ■ 




.^17 / 

/ 


.33- 


.10 


.06 


.17 


-.05 


^38 ^ ^ 




/ 

/ 
















.39 


.25 


.'32 


■ .06 


.64 


-.07 


-.30 




.03 


.10 


.14 


.06 ' 


.37 


-.06 


.72 ■ 





.06 .80 -.06 -.34 
.05 -.49 -.07 .57 
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Table Xr-6 ' 



\ 



Correlations and Critical Ratio;5 — ^ 
Aptitude Tests with Work .Sample, Overall Performance 
Inventory Management Specialists 



Correlations xy " ^ 

- ' Mexican- 
* Aptitude Tests^* BJ.ack American Caucasian 



Critical ratios 

Caucasian/^ . Caucasian/^ 
Black .Mexican-American 







• 


• 


CR 

yy 


CR» 

XX 


CR 

yy 


GR . 

XX 


Ni&ber 


.17 


- 

!36 - 


.34 


.64 


as 


-.08 


-.15 


^HiddenVFigures 


.21 


.29 


.30 


.56 


.50 


-.05 • 


.12/ 


Vocabulary 




.41 


.37- 


.61 


.53- 


-.07 


/<74 


6b j ec trHumb er 


.04 • 


.20 


• .06 


.61 


-.01 




-.56 


Letter Sets 


.28 


.49 


.29 


.62 


:50 


.Xo7 


.24 


Nonsense 
Syiiogism's 


.29 


.38 


.13 


.56 




' . -.06 . 


■• .38 


Subtraction ^ 
Multiplication . 


. .08 


.37 

/ 


"!l3 


.65 


.59 

> 


-.08 


.29 . 


Extended Range 
Vocabuiairy 


.28 • 


.58 / 


.32 


.64 ■ 


.36 


-.08 


.63 


Necessary 
Arithmetic 


.33 


.60 


-.35 


.61 


.83 


-.07 


.35 


Operations 
















Inference 


.39 


.56 


.34 


.59 


.50 


-.07 


.47 



0 
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. . Chapter Xll 

^ Conclusions ,and Inferences^ 
The preceding chapters have described in detail the statistical 
treatment of the data obtained in this study and the conclusions 
.denived^ This chapter will recapitulate the moire important findings 
arid draw some of the inferences that follow from them«# 

4 

Conclusions 

^ ■ 1« There is little in the data to support the hypotheses of 

. dif ferentialjvalidity for the wide variety of tests studied 
--^-^^""^^"''^ • for the* ethnic groups included;;, in this study. Tests which 
were valid for one ethnic^gfoup were also valid for the 
other ethnic group(s)#^ This Keld true When the tests were 
J • 'used to predict all three kinds of criterion measures, 
2, Tests valid 'against one kind of criterion were generally 

valid against other criteria also, 
3» When supervisors' ratings are used as the criterion, there 
, is little difference in the regression lines for different 
ethnic groups^' i.e,, a particular test score predicts the 
same level' of' job performance for all ethnic groups. 
4. When work samples or job knowledge tests are used as 
' ' criteria, there usually are differences in the regr'^e^sion 

li-nes^ between majority and minority ethnic groups. In 

• - • *%' 

these instances, a given test score is associated with 

higher job performance for the Caucasian group than for 

'I * * 

the other two groups. / 

5'. When test scores ^re combined by a multiple regression ^ 

" -423- ^ • 

^ X' - - ■ 



-424- 

equation, there is no practical loss in predictability when 
the equation developed for one ethnic group is used for 
prediction of criterion scores for -the other ethnic groups. 
Supervisors' ratings were affected by interaction of ethnic 
group membership of the r'ater with the .ethnic group member- , 
ship of the ratee. Raters tended to give highjsr r^atings to 
ratees of their own ethnic group. Ratings of Black job- 
incumbents by Black supervisors had hi*gher corfMation with 
Other measures than, did ratings of Caucasian job incumbents 
by Black supervisors. In contrast, ratings of Caucasi^fn job 
incumbents by Mexican-American supervisors. had higher cor-y 
relations with other measures than did their ratings of 
Mexican-Americans. Ratings by Caucasian raters of all three 
ethnic groups correlated about equally well with oizheir 
measures. , . 

There is-vno substantial difference in background or experi- 

• - "." , 

ence variables, for the different ethnic groups. Possibly 

_- • - - ""./ 

for this reason, use of moderator variables such as length 

/ 

'of experierjce or amount of education did not produce signifi 
cant improvement in predictability. 

Mean scores ior minority groups on aptitude tests are 

generally about one-half standard deviation below the mean 
t * , * 

scores for Caucasians.. There are, however, a few instances 

/ 

* ^, 
where the mean for Mexican-Americans is above the mean for 



Cencasians. \ 



\ 



Mean scores for minority groups on job knowledge tests and 
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^ ^ work samples are similarly about one-half standard deviation 
below the Caucasian mean score. There is one instance where 
the mean score for Mexican-Americans is above the mean score 
for Caucasians. 

10. In .cohtrast to the dv^fferences on tests and work samples, 
^"^ • , the means of supervisors',^ ratings for minority groups are 

.. ip .very close to those fo.r Caucasians. 

11. Factor analyses of the test and criterion measures shox>r v^'iry 

similar patterns for all groups. ^ 

/ ' 
^Inferences and Implications 

' \ ^ . 

It is perhaps appropriate to comment on some of the inferences 

• . / 

or implication^ which follow fronf the. data which have been presented. 
Some of these comments reflect or -overlap those made by speakers at 
the Invitational conference previously mentioned (Crooks, Ed. 'l972), 
although no effort will be made to recapitulate all of these comments 
here. 

- - In view of the evidence fxom this study and other evidence pre- 

I \) . ' * 

sented by Boehm and Ruch, appears that differential validity, if 

• i " ' 

not entirely a statistical artifact where it does appear, is at best 
. an isolated phenomenon. Furthermore, tfhe use of different regression 
lines for different ethnic groups usually will operate to reduce rather 
tha'n enhance employment opportunities for minority groups. 

Evidence of the effect of rater-ratee ethnic group interaction 
on ratings makes the use of supervisors* ratings (or similar judgmental 
^ variables such as giades) as criteria of job performance a somewhat 
dubiojijs proceeding where different" ethnic groups are involved. 



erJc 



sop 



However^ it "is reassuring' to find, in tjiis s1:udy at least, that tests 
selected to predict supervisors' ratings for one group also made valid 
predictions of more "solid" c^teria such as job knowledge tests and 
work samples for all grOupsi 

It. appears evident thit--iche recognized problem of minority under-^ 
representation J in technical, professional, and managerial j9bs — or, in ^ 
.many instances, outright unemployment — will not be solved by the differ- 
ential prediction approach, since this .approach more" oftWn than not will 

reduce rather than increase the proportion of minority applicants employed. 

- * ' ' ."^ ' ' \ ' 

The question s*till remains, "What can be done to solve tliis problem?"* 

Answers — or l^ypoth^ses about the answers — to that question go beyond 
the scope of this "research. Yet the question is tpo important to ignore 
or evade. One suggesti^on (3ro;iTi, 1972) is that employers use a low 
.cutting score and provide* additional training to those who score low. 
This is ^ reasonable suggiestion where, employers can affoi;d to, inve;St 
substantial traiffing and are not bound by legal requirements to hire rrom 
the top of a qualifying list. If test scores are used to predict subse- 
querit jbb .performance, vh^ should they not also be usTed to predict how 
much or what kind of training the new employee needs? 

Tliere is reason for some cautious optimi^, in the fact that back- \ 
, ground variables and test scores (do seem to, have the same meaning for 
individuals from different ethnic groups^^lt appears ppssible that if 

v.- # ■ ^- -f 

all grruj ^»have. equal opportunity in all aspects of their l;ives — and it 
. is evident that so far minority groups have not had equal opportunitj^T^ 
then eventually the differences in test scores between groups* will 
become inconsequential. 
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An Attempt to Minimize Sources of Bias in Supervisors' Ratings 
by Use of a Standard Stimulus/ 

In an effort to minimize known sources of bias in supervisors* ratings 

such as halo effect, rater leniency, and social desirability set, .and 4:o • 

focus on the behavior to be rated on each^ scale beyond the definitions and 

anchors given, a fictitious incumbent was created for each occupation 

Studied. On the facing page for each scale, the typical behavior of this 

ihdumbent was described as it related to the aspect of performance defined 

by the .scale. On the basis of this behavioral description, the supervisor 

was asked to rate the fictitious person on the scale before rating the real 

incumbents he or she supervised. It^was theorized that ratings^ made on 

such a standard stimulus would not only provide a benchmark of the super- 

visor's rating behavior as a basis for adjusting for possible bias in other 
/ ' ■ ^ / ^ # 

ratings, but in addition might influence the rater to consider typical job 

^behavior more carefully in rating those actually supervised. 

I A detailed description of the procedure and the findings in the 
Medical Technician study may be found in Parry "and Mahoney (1970X, v;hich 
will only be summarized here. *A. similar analysis for Cartographic Techni- , 
/cians was also made. Because the findings in these two studies as to the^ 
desirability of adjusting ratings on the basis of benchmark -rating behavior 
were somewhat inconclusive, no such detailed analysis was made for 
Inventdry Management Specialists. However, the fictitious incumbent was 

* ' ' ' . ■ 

again used in the rating procedure as a focus and further guide for the 
supervisors in rating their "real" incumbents. 

the descriptions of the fictitious persons' were made as realistic as 
possible by referring to them by name (Otto Analyzer, Medical Technician; 
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Mark Bench, Cartographic Technician; and Mel Stripp, Inventory Management 
Specialist), Although there was no effort to make the character of Otto 
Analyzfer consistent from scale to scale in the Medical Technician study., 
* ah attempt was made in the other two occupations to describe Matk Bench 
and Mel Stripp across scales with consistent and believeable attributes, 
as possibly having more face validity* An overall rating was not asked 
for on the Medical Technician study, but was eTicited fo?; the other two, . 
Examples of Mark Benches descriptions for three scales are given in ^ ^. 
.Appendix Figure II-l. ' 



Analysis of the Data 

A mean rati-ng for the imaginary person was obtained for each of the 
scales where he was described, based on ^e number of supervisors who , 
ratied him. These mean ratings were assumed to be his "true" ratings. The 
means and standard deviations fdr* these* ratings* for each scale for the . 
Medical Technicians are given in Appendix Tabl-e II-A, and for the Carto- 
graphic Technicians in Appendix Table II-B, The possible ratings for each 
scale ranged from 1 (lowest) to 9 (h^^hest). As may be seen from the 
variations in means across scales in these two tables, the supervisors did 
seem to be attending to the behavioral descriptions given. The ratings 
that supervisors had assigned tfo their "real" people were then adjusted, 
» by scale, according to how each supervisor had rated the standard stimulus 
in relation to his mean ratings. For example, if the supervisor rated 
Mark Bench 8 and his mean rating for a given scale was 6,4, 1,6 was sub- 
tracted from the rating the s\iperviso.r gave each of his technicians on 
this scale. For any scale, ift:the imaginary person had not been rated by 
tnbfsupervisor, no adjustment could be made, and the supervisor's original 
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T^tfings for that scale were used^ In cases where individuals had been * 
-rated by more than one supervisor, an average of the adjusted ratings was 
obtained^ Appendix Table II-C shows the "means and standard deviations, 
by scale, for the ratings received by Black and Caucasian Medical Techni- 
cians. For the Black technicians, the adjusted mean ratings were higher 
than'the originals on five scales and Jower on two. For the Caucasian 
technicians, the .adjusted mean ratings %^eve higher than the originals on 
four scales,^ lower on two, and equal on one. Appendix,.Xable II-D shows 
the means and standard deviations, by scale, for the ratings received by 
Black, Mexican-American, and Caucasian Cartographic Technicians. For 
the Black technicians, the adjusted mean ratings were higher than the 
orfginals on one scale and lower on six. ^ For the Mexican-American techni- 
cians, the adjusted mean ratings were higher than the originals on two 
scales and lower on five. For the Caucasian technicians, the adjusted 
mean ratings were higher on five scales and lower on two. For all three 
ethnic groups, across both occupations, the variance of the adjusted* 
.ratings was greater than that of the unadjusted ratings for every scale 
but two. Thi^ may partly be due to the fact that the method of adjustment- 
used increased the range of the scales. For example, if 3 were being 
added to all ratings given by a supervisor on a particular scale, and a 
technician had originally been given a 9, he would'nbw receive' a 12. 

The adjusted ratings for each technician' were theni used as new 
criterion measures. The scores from the aptitude test battery administered 
to the' technicians were correlated with the adjusted ratings and.MCorapared 
with the correlations obtained between the aptitude tests and ^the un- 
adjusted ratings. Appendix Table II-E presents the correlations, for Black 
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Medical Technicians, Appendix Table II-F for Caucasian Med^ical Techni- 
cians^ Appendix Table II-G for Black Cartographic Technicians, Appendix 
Table II-H for Mexican-American Cartographic Technicians, and Appendix 
Table Il-r for Caucasian Cartographic Technicians. For the Black 

) < ■ . ; . 

MeHical /Technicians the correlations based on the adjusted ratings were 

* • . - r 

lower than those based on the unadjusted ratings in 65 percent of the 

cases, unchanged in five percent, and higher in 30 percent. For the 

Caucasian technicians they were lower in 70 percent, unchanged in eight 

percent,* and higher in 22 percent. The differences in either direction 

were^small, ranging from .00 to .11. No pattern of change was evident, 

either by rating scale, aptitude test, or r^Cil. For the Black Carto- 

graphic Technicians the correlations based on the adjusted ratings were 

lower than those based on the unad^sted ratings in 44 percent of the 

cases, unchanged in eight percent, and higher in 48' percent. For the 

/ ^ 

Mexican-American technicians they were lower in 57 percent ,gf the cases, 
unchanged in 13 percent, and higher in 30 percent. For the Caucasian 
technicians they were^lower in 53 percent, of the cases, unchanged in 12 
percent, and higher in 35 percent. Again, the differences in either 

direction were small, ranging from .00, to .15. . 

' / 

Job Knowledge Test scpres were also correlated with the adjusted 

^' , . ^ • ^ 

supervisors* ratings (Appendix Tables jE^-E to II-I) for both occupa- 
ti-oriSi' The correlations were lower than had been obtained-^with * the 



unadjusted^atings with only one exception (Caucasian) for ^t1ie Medical 
Technicians and five exceptions for the Cartographic^iechni^cians - (four 
Mexicaifr^American and* one Black) . If one consid^s the Job Knowledge 




Test as another criterion measure, it is^^ss related to t\i^^ aspects 
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of job performance which the adjusted ratings measure « than ^irt is to those ., 
which the unadjusted ratings measure*^ , 

When- considering the correlations between the Job Knowledge Tes^^nd 
the^ job knowledge ratings, the difference between the correlations based 
oji adjusted and unadjusted ratings is greater for the Black Medical Techni- 
cians than for;|||he Caucasian Medical Techn^cl.ans^(a difference of ,11 
versus .02). For both ethnix: groups^y^e correlation ^>f Job Knowledge 
Test scores with unadjusted job-^Icr^l^^ge^ratings is higher than witl>. 
adjusted ratings. For tjie Cartographic Technicians the differences are 
less marked (.OS^fjot the , Caucasians versus .05 for the Mexican-Americans 
and .04 for the Blacks). For the Blacks, the correlation of Job Knowledge 
Test -scores witlT'unadjusted job knowledge ratings is lower than with 
^adjusted^atings, and for the- Mexican-Americans and Caucasians it is 



^^high^r with unadjusted than with :adjusted. 
Discussion 



Why did adjusting the ratings in general Jail to improve their cor- 
relations with thc^ aptitude tests and the Job Knowledge Test? The reasons 
are not entirely clear. Apparently adjusting* the technicians* ratings on 
the basis of pooling the judgments of a large number of supervisors 
resulte'^d in the addition of more random variation instead of correcting 
for differential preferences in rating behavior.^ 

One possible explanation is that the descriptions of the various 
aspects of the fictitious person's job performance were too specific and 
limited. If too little information was given on which to base the ratings 
of his performance, then^ adjusting, all the ratings given by the : ipervisors 
according to their ratings of him could not be expected to improve the 
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correlatrions between test scores and ratings. It may be easier for a 
supervisor to provide ratings for his technicians based on many relevant 
behaviors and incidents than it is to rate a simulated individual on the 
basis of only a paragraph description. The ratings of this person may, 
therefore, include more error variance than the ratings of the "real" 
people. 

Another possible explanation is that the supervisors did not take 
the task of rating the f ictive incumbent as seriously as they did fating 
their own personnel. If the raters exercised less care in rating him, 
this could explain the lower correlations found when adjusted ratings 
were' used. 

. For the seven scales used from each of the two studies, adjusting 
the supervisors' ratings on the basis of their ratings of an imaginary 
subordinate did not generally improve the correlations between the 
aptitude measures and the ratings trf job performance. It is possible 
that another type of analysis, .such as a correction for each supervisor* 
. across scales rather than for each scale, would produce different 
f r^esults, although this would seem unlikely in view of the rather limited 
findiflfes of the present analysis. 

It is felt that the standard st'imulus may still have served the 
other purpose for which he was included in the rating .scales, that of 
helping to orient the supervisor toward the qualities of each job 
dimension to be rated. Also, it may be that having to rate a fictitious 
person, where the supervisor knew that his responses could be compared 
with those of other supervisors, tended *to make him more attentive to ^ 
'•the standards he was using. While taking account of the definition 
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given for the scale and the description given of the fictitious indi- 
vidual, he may hafve been led to exercise greater care in rating the 



, actual incumbents. 



/ 



ERIC 
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Mark Bench reajly loves maps. He collects old maps • 
and books about cartography and likes to show each new 
find to his less enthusiastic co-workers. He spends a 
lot of time' reading and studying about maps and mapping, 
but it is not always relevant to the work he is doing. . . 
He is inclined to take more interest in his work if he 
can spend time figuring, out how to use a new piece of 
equipment, assembling source material, making elaborate 
plans, and looking up specifications. He would like to 
leave the routine compiling to others, although it is 
part of his job. 



(Interest) 



When faced with an assignment requiring fine detail, 
crowded, features, or the use of small equipment such as 
gravers, Mark Bench seems to be "all thumbs." His con- 
touring is sometimes so messy that it looks like a pile 
of wet spaghetti and he has to. .re-do it several times 
before it is acceptable. His eye-hand coordination is 
off just enough so that he has difficulty putting stick 
up just where he wants it. On tasks where less precision 
is required, he produces quite acceptable work. 



(Dexterity) 



Using tt^. descriptions of Mark Bench and his work 
given for the previous scales, please give him an* over- 
all rating. This may not be a true evaluation of him 
as a cartographic technician, because you have never 
actually seen anyof his work or have never met him in 
person. From your impression of him, jrate him on this 
scale along with the other people in your group. 



(Overall Psrformance) 

Appendix Figure II-l Examples of Behavioral Description 

for Fictitious Cartographic Technician 
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. ^ Appendix Table II-A » ^ 

Means and Standard Deviations for the 
Ratings Givei^ to Fictitious Medical Technician by All Supervisors 

N=200 



Rating Scales 


Mean 


S,D. 


Organization 


6.4 


2.0 


Interest 


6.6 


2.3 


Learning Ability 


4.4 


1.3 


Job Knowledge 


4.3 


1.5 


Technique 


1.9 


1.2 


Need for Supervision 


5.3 


1.4 


Communication 


4.3 


1.8 
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u ^ Appendix. Table II-B 

Means and Standard Deviations for the 
Ratings Given to Fictitious Cartographic Technician by All Supervisors 
(TOPOCOM and Coast & Geodetic Survey) 
N===130 



Rating §cales„ 
Interest 

LeatTiihg Ability 
Job Knowledge 
Dexterity 

Need for Supervision 
.Perseverance 
Overall Rating 



Mean 

6.0 

4.6' 

5.2 

1.7 

3.3 

3.8 

4.2 



S.D. 

1.9 

1.4 

1.3 

1.1 

1.7 

1.4 

1.5 
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rRatliig; 
Scaiest 



Appendix Table II-C 
Means and Standard Deviations fo.ir Original and 
Adjusted Supervisors ''Ratings of 
Black and Caucasian Medical Technicians 

f Black 
(N=166) 



Cauc^ 



,sian 
5) 



Mean " S .D, ^ 

prig. Adji Orig. Adj.. 



Mean S.iD. 
Orig;. Ad jT\ . prig. M ji 



,Interest f 
Learning ^Ability 
Job Kncwiedgfe 
Technique 
Need for Supervision 
Communication ^ 



5.75 


5.73 


1.82 


2.26 


5.99 


6.12 


1.79 


2.33 


5.57 


5.88 


1.78 


' 2.43 


5.9.0 


6.08 


1.74 


2.46 


5.79 


5.82 


1.88 


2.07 


5.96 


5.82 


1.77 


2.05 


5.23 


5.09 


1.88 


2.01 . 


- 5.30 


5.55 


1.74 


2^08 


5.90 


5.97 


1.-73 


1.-73 


6.01 


6.06 


1.73 


1.86 


5.72 


5.82 


1.97 


2.08 


• 6ilO 


5.98 


1.86 


2.07 


5.49 


5.85 


1.81 


2.13 


■ 5.75 


5.75 


1.75 


2.06 
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Appendix Table .II-E 
Correlations Between Tests and Supervisors' Ratings 
, . for Black Medical technicians 

Original Above - Adjusted Below 
'§ (N-166) 

« J 



ors' Ratings 



a 

U 

i 

d 

o 

4J 
• O 

•s. 

•§ 

CO 



u 

O 

o 
> 



0) 
0) 

3 

d 



u 

J. 

w 



^ 5s 


' d 


o 


^ 4J 


w 


t4 


•H 


U 


>4 


0). 




4J 








De 


O 




u 


0) 




00 




^d 


1 






' s 





d 
o 

4J 
0) 
r-l 

a« 

0) 



I: 

0) 

3 
4J 
U 



00 

d 
a< 

CO 

PL4 



-CO! 

0) 

00' 

0): 



Organization 



^Ihterest 
Learning Ability 
Job Khowledga 
Technique' 

Need for Supervision 
Communication 



.32 

.27 ■ 

.29 
.20 

^36 
.37 

.32 
.25 

.28 
.25 

.27 
.27 

.23 
.25 . 



.10 
.08 

.10 
.11 

.20 
.14 

i20 
.18 

.15 
.08 

.10 
.07 

.21 
.19 



.03 
.04 

.05 
.12 

.02 
. .00 

.07 
,13 

.04 
.02 

.03 
.01 

-.02 

\03 



.19 
.23 • 

.17- 
.20 

.34 

•;30 - 

.33 
.25 

.23 
.21 

.25 
.21 

.22 
.19 



.09 
-.01 

.04 
.02 

.20 
.13 

.09 
.04 

.08 
.04 

.05 
.02 

.06 
.05 



.15 
.08 

.07 
.06 

.2'6 
i26 

.18 
.16 

.17 
.11 

.10. 
t^O^ 

.14 
.15 



.06 
.02 

.03 
.04 

.18 
.li 



-.07 .01 
-;12 -.01' 



-.07 
-.09 

.06 
.07 



.14 -.02 

.06 -.07 

..08 -.06 

.03 -.02 



.04 
.06 

.12 
.09 



-.00 
.01 

-.05 
-.03 



.00 
-.02 

i 

.21 
.18 

.09 
.06 

.07 
.09 

.09 
.08 

.11. 
.06 



.45 

'% 

.38 
.20 



.58 
.53 

.58 
.47 ' 

.49 

.44, 

.43 

;38 

.50 
.35 
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Appendix Table II-F 
Correlations Between Tests and Supervisors* Ratings ^ 
for Caucasian Medical Technicians 
Original Above - Adjusted Below 
g (N=285) 






.15 


.01 


..03 


.16 


.16 


..05 


.14 


.14 


.10 


.27 


.Organization 


.14 


-.02 


.06 


.15 


.08 


.03 


' .10 


.09 


.09 


.25 




.12 


.06 


.04 


.15 


.12 


.07 . 


.06 ' 


.12 


.10 


.22 


Interest 


.06 


.01 


-.04 


.08 


.01 


.02- 


-.02 


.04 


.08 


.17 




.24 


.07 


.12 


.30 


.26 


.16 


..19 
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.28 


.35 


Learning Ab'ility 


.19 


.09 


.09 


.26' 


.18 


.07 


.18 


.22 


.23 


.29 
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r.Ol 


.12 


.10 


-.01 
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.12 


.35 


Job Knowledge 


.06-' 


.07 
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-.07 


-.02 


.06 


.06 


.33 
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.11 


.06 


■ .06 


.16 


.17 
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.14 


.19 
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.23 


.08 


.03^ 


.05 


.17 


.15' 


.06 


.13 


.22 


.14' 


.18 
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.27 


Need for Supervision 
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.05 


.03 


.07 


-.01 


-.08 


.00 


.01 


.04 


.30 




. .06 


.17 


.08 


.13 


.06 


.01 
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.05 


.10 
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.02 


.18 
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.10 
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Appendix^ Table III-A 
INTERCORRELATIONS OF PREDICTORS WITHIN ETHNIC GROUPS 
Medical Technicians 



1. 

1, 

3, 
4. 

5. 
6. 
7. 
8. 
9. 



■ • (Black 


- first 
N-168 


line; 

- 


Caucasian - second line) 
N-297 

* 








.test 


1,. 


2. 


3. 4. 


5. 


6. 


2. 


8. • 


9. 


Subtraction & 
Multiplication 


1.00 
1.00 


.22 
.08 


.08'* .43 
.11 .37 


.13 
.19 


.44 
.44 


.22 
.08 


.22 


.16 
.08 


Extended' Range 
Vocabulary 






.10 .29 
.15 .33 


-.08 
-.01 


.18 
.03 


.18 
.02 


.15 
.08 


.20 
.22 


Hidden 
Figures 






-- .26 
.40 


.26 
.37 


16 
" .33 


.30 
.38 


.14 
-.22 


.15 
.42 


Necessary 

Arithmetic 

Operations 








.10 

.39 


.39 
.40 


.36 
.42 


.34 
.36 


.43 
.56 


Pin- 
Dexterity 






* 




.45 
.48 


.21 
.50 


.10 
.26 


.19 
.47 


Number 
Comparison 












.42 
.35 


.18 
.29 


.^3 

,32 


m 

Gestalt 
Completion 














.■25 
.20 


.44 
.52 


Picture- 
Number 
















V30 
.13 


Paper 
Folding 






/ 













/ 
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Appendix Table III-B . 












- 






INTERCORRELATIONS OF PREDICTORS WITHIN ETHNIC GROUPS 


% 










« 


Cartographic Technicians 


- TOPOCOM Sample 












(Black 


- first 


line; Mexican-American - second ■ 


line; Caucasian 


- third' line) 








N-lOl 


N«99 








N»=241 










Test 


1. 


2. 3. ^ 4. 5,- 




2* 


s. 


9^. 


• 10 . 


11. 


> 12 . 


13. 


1. Coordination* 


1.00 


.06 -.15 .21 .14 


.08 


.17 


.12 


.33 


.20 


.29 


-.12 


.18 " : 




1^;00 


.25 .05 .12 .26 


.18 


.10 


.22 


.40 


.20 


.36 


.03 


.12 


I • 


1.00 


.21 -.03 .13 .24 


.16 


.25 


.23 


.33 




.20 


-.01 


.20 


;2. Hidden 




..32 .12 .43 


.47 


.48 


.53 


.39 


.40 


.45 


.34 


.44 


\ ' Figures 




.03 .16 .35 


.34 


.40 


.59 


/.49 


■ .48 


.51 


.08 


' .53 




.20 .26 .31 


.28 


.34 


■y 


.41 


.Jo 


.34 


.15 


.31 J 


i3* Vocabulary 




..13 .OS 


.22 


.19 


' .it 


.06 


.26 


.11^ 


.88 


.46 : 






-.02 .13 


.14 


.03 


.:i6 


.04 


.30 


MO' 


.77 


.24 ^ 




■ 


.13 • -.01 

* 


.21 


.11 


.15 


-.02 


.31 


.09 


..82 


.34 


4. Object- 




.25 


.27 


.25 


.32 


.24 


.29 


.30 


.12 


.28 


Niiinber 




-.07 


.12 


. .17 


.12 


.08 


.09 


-.17 


.04 


.09 ^ 






.03 


.20 


.23 


.16 ■ 


.14 


.16 


.23 


.14 


.19 : 


5. Card 




— 


.42 


.50 


.48 


.40 


.35 


.53 


.11 


.38 


Rotations 






.36 


. .33 


.40 


.49 


.31 


.39 


.05 


.32 






■ 


.26 


• 46 


C / 

.54 ' 


c c 
.55 


.'»3 


.48 


-.06 . 


.26 


CS Arithmetic 








.53 


.54 


.34 


.57 


.53 


.26 


.63 : 










". 31 


.40 


.60 


.32 


.51' 


.09 . 


.49t ; 










A9 




. 


LL 


.41 


.11 


.53 ; 


"7. Map Planning 










.56 


.53 


.51 


.56 


.24 


.50 ' 












.37 


.43 


.34 


.28 


.03. 


.31 






• 










45 


.44 


.01 


.^1 


8* Surface 













.44 


.57 


.46- 


.27 


.34 


Development 












.55 


.54 


.43 


.16 


.62 














.49 


.56 


.44 


.11 


.49 


9. Maze Tracing 














.33 


.54 


.10' 


.34 ' 


Speed 














.40 


.68 


.01 


.46 
















.42 


.55 


-.07 


.30 


10. • Following Oral 




• 












.48 


.34 


.66 


Directions 
















.26 


.29 


.61 


















.37 


.29 


.59 


m 


















.08 


.41 


11. Identical 
















— 


Pictures 


















.04 


.40 




















.01 


.23 


12. Extended Range 


- 
















— 


.49 \ 


Vocabulary 




















.27^> ^ 






















.30 


13. Necessary 




















— 


O litnetic 
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Appendix Table III-C 
INTERCORRELATIONS OF PREDICTORS WITHIN ETHNIC GROUPS 
Cartographic Technicians - Coast & Geodetic Survey Sample 
. (Black - first line; Caucasian - second line) - 



N=38 



N=»^51 



Test . 

' !• Coordination' 

\ 

' ?• Hidden ^ 
Figures 

3. Vocab"ulary 

4. Object- 
^ - Number 

5., Card 
* Rotations 

• 6* CS Arithmetic 
?• Map Planning • 



8* Surface 

Development 

9. Maze Tracing 
Speed 

10. Fjllowing Oral 
Directions 



1. 

1.00 ^ 
.1.00 



1. 


•3. 


4. 


5. 


6. 


7. 


8. 


9. 


itf: 


11- 


li- 


'13. 






















02 


.21 


.02 


.46^ 


.33 


.36 


'.19 


.47 


.52 


.49 


.14 


.29 ' 


29 


.13 


.10 


.18 


. .00 


.25 


' .29 


.29 


.26 


.37 


.02 


.10 % 


~ 


% .30 


.53- 


.54 


.45 


.31 


.70 


.'48. 


.35 


.42 " 


.41 






-.10 


.02 


' ".41 


.38 


.44 


.63 


.58 


.60 


.56 


-.09 


.58 t 




% 


.12 


\ .34 


.46 


.53 


.45 


.21 


.54 


.14 


^.86 


.39 1 






.09 . 


.06' 


.21 


.08 


.08 


-.08 


.13 


-.14; 


.79 


.20 








' .37 


..40 


.38 


.46 


.74 


.18 


.57 


.25 


A .09 








.10 


\29 


^.33. 


.16 


.29 


.36 


.09 


.16 


; -.20 




\ 






.52 


'.51 


.65 


.62 


.58 


.58 


.29 


.56 : 










.51 


.57 


.67 


.48 


.52 


.60 


.03 


. .60 












• 54 


.49 


.50 


.70 


'.53 


.47 


.61 












.56 


.46 


.49 


.73 


.45 


.17 


.77 




-* 










.44 


.48 


.48 


.53 


.16 


.50 














.65 


.61 


.69 


.64 


.1.2 


.56 
















-.40 


'.52 


.51 


.48 


.55 
















.67 


.66 


'.62 


.08 


'.63 


















.45 


.56 


.10 


» .48 


















.55 


.68 


-.04 


^ .55 
















* 




.43 


.54 


.68 




















.58 


.14 


^__.69 



11. Identical 
Pictures f 



12. Extended Range 
Vocabulary 

13. Necessary 
Arithmetic 
Operations 



\ 



.10 
-.19 



.33 
.22 
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Appendix Table III-D 

INTERCORRELATIONS OF PREDICTORS WITHIN ETHNIC GROUPS 
* ♦ 

Inventory Management Specialists 
(Black - first line; Mexican-American - second line; Caucasian - third line) 



N=112 



N=72 



N=200 



Test 

Number 
Comparison 



Hidden ^ 
Figures 



Vocabulary 



1. 

1:00 
1.00 
1.00 



2. 

.38 
.17 
.34 



Object- 
Number 



Letter 
Set^ 



Nonsense 
Syllogisms 



Subtraction & 
Multiplication 



.Extended Range 
Vocabulary 



9. Necessary ^. 
Arithmetic 
Operation^ 

10* Following Oral 
Directions 



11. Inference 



3. 

.21 
.23 
.17 

.23 
.42 
.33 



4. 

.07 
.08 
.15 

.16 
.02 
.12 

.09 
.16 
.09 



5. 

.41 
.31 
.47 

.50 
^.34 
.52 

.55 
.34 
.47 

.19 
.22 
.22 



.12 
.13 
.10 

.25 
.17 
.27 

.46 
.36 
.31 

.02 
.08 
.10 

.38 
.33 
.45 



.41 
.45 
.54 

.19 
.30 
.11 

.22 
.27 
.06 

.05 
.2G 
.17 

.21 
.35 
.34 

.16 
.16 
.08 



8. 

.07 
♦ 23 
.13 

.21 
.36 
.36 

.88 
.73 
.85 

.03 
.22 
.11 

.50 
.35 
.50 

.53 
.46 
.36 

.12 
.36 
.04 





10. 


11* 


12* 


• 2j 


« Jj 


.11 


♦ 36. 


.33 


♦ 29 


.13 


.25" 


.41 


.38 


.27 


.33 


.JJ 


.47 


• 2o 




.45 


.42 


.36 


.47 


.47 


.42 


.34 


.46 


.63 


.43' 


• 69 


♦ 76^ 


.50 


.36 


.45 


♦ 65 


.50 


♦ 38 


.57 


.69 


.13 


.16 


.20 


.18 


.18 


.07 


:r8 


.26 


.17 


.18 


• 13 


♦ 22 


.64 


.63 


.53 


♦ 66 


:5i 


.46 


.38 


♦ 60 


.62 


.56 


.60 


♦ 71" 


.50 


.47 


.56 


.55 


.45 


.61 


.30 


.54 


.44 ' 


.32 


.43 


• 50 


.32 


.33 


.11 


.32 


•42 


.42 


.33 


.47 


.40 


.20 


.21 


.27 


.56 


^43 


.69 


.76 


• S'i 


.33 


.52 


.71 


,49 


.43 


.62 


.73 




.61 


.63 


.68 




,59 


.59 


^72 




.59 


-60 


.76 






• 56 


♦ 59 






.39 


.53 






.50 


♦ 60 








♦ 74 








.69 








.74 
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Appendix Table V-A 
Correlations of Aptitude Tests with Supervisors' Rating^* 
and Job Knowledge Test by Ethnic Group 
Medical Technicians 
(Black - first, line; Caucasian - second line) 

























Test- 






^3 


\ 


^5 


^6 


h 


^8 


^9 


■ JKT 


f 

Subtraction & 


.35 


. 32 


.29 


.36 


.32 


.28 


ill 


.23 


.•29 


.34 


Multiplication 

- 


.24 


:i5 


.12 


.24 


.09 


..11 


.05 


.06 • 


.13 


.23 


Extended Range 


.11 


. 10 


.10 


.20 


.20 


.15 


.10 


.21 


.09 


.32 


Vocabulary 


-.00 


.01 


.06 


.07 


.13 


.06 


.05 


.17 


.06- 


.27 


Hidden Figures 


.04 
.16 


' . 03 
.03 


.05 
.04 


.02 
.12 


.07 
-.01 


.04 
.06 


. 03 
.03 


-.02 
.08 


.02 
.05 


.15 
.14 


Necessary 
/iu X uniuc L X c 
Operations 


.29 
.29 


.19 . 
. 16 


.17 
.15 


.34. 
.30 


.33 
.12 


.23. 
.16 


.25 
.09 


.22 
.13 


.17 
.16 


.46 
.34 


Pin-Dexterity 


.14 
.26 


.09 
.16 


.04 
.12 


..2oJL 

.26 


. _..05._ 
.10 


. 08^ 

.17 


-...05 
.07 


.06 
.06 


.10 
.12 


.28 
.17 


Number* 


.16 


.14 


.07 


.26 


.18 


.17 


.10 


.14 


.17 


.23 


Comparison 


.16 


.05 ■ 


.07 


.16 ■ 


-.01 


.08 


-.01 


.01 


• .04 


.14 


Gestalt 


.14 


.06 


.03 


.18 


.14 


.08 


.04 


.12 


.05 


.25 


Completion" 


.23 


.14 


.06 


.19 


.03 


-.14 


.03 


.03 


.11 


.ii 


Picture- 


-.03 


-.07 - 


•'.07 


.06 


-.02 


-.06 


-.00 


-.05 


-.02 


.21 


Number 


.22 


.14 


.12 


.20 


.06 


.19 


.09 


.05 


.15 


.16 


Paper 


.13 


.00 


.00 


.21 


.09 


.07 


.09 


.11 


.08 


.■22 


Folding 


.26 


.10 


.10 


.28 


.12 


.15 


.06 


.10 


.11 


.21 



*Rating Scales 

= Flexibility Rg = Technique 

R2 - Organization R^ = Low Need for Supervision 

R^ = Interest Rg ~ Communication 

R^ '= Learning Ability R^ « Overall Performance Rating , 

R^ = Job Knowledge 

JKT = Job Knowledge Test * 
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mmm in<no\ c^c^c^ mcorH vocost osrrH o\o\ vr' 

r^C^^O Cn^C4 e>40C>4 OOrH C>lsrCO COCOi-l ,C4C0C>I» 

iHcooo o\oco rHc^sf srmoo c^mm sroir^ r^^r> 00 

i-lC>40 i-lOi-l C^OO ?nMC4 cncnO C>4C>4rH 

• * I r * 

oomvo r^sroN oooco vosrvo m r-i <t tnmi-i o»-iin 

O CO C4 O ,rH. rHOO rHC0C>* ,rHC>4O" C^COrH 

C0VOC4 voo\o\ vovooo cnco . vovocn c>40oco c>4inso. 

Oi-li-l C^C^i-l i-lOi-l 000 OC>4 C^W i-l i-l 1-1 .1^ CM 1-1 

t - * 

vo 00 tH 1-imo vocMON 00 ststcn eg <t cn m c>4 o 
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J ' Appendix ^Table V-C 

Correlations of ^Aptitude" Tests with Supervisors' Ratings* by Ethnic Group 
Cartographic Technicians (Coast & Geodetic Survey) 



Test: 

Coordination 
Hi'dden Figures. 
Vocabulary 
Object-Number ^ 
Card Rotations 
CS Arithmetic 

Map Planning 

Surface 
Development 

Maze Tracing 
Speed 

Following Oral 
Directions 

Identic^ 
Pictures 

ExtendLd Range 
Vocabulary 

\ 

Necessary ' 

Arithmetic 

Operations- 



R = 

.^2 = 
R = 



(Black - 


first' 


line; Caucasian • 


- second line) 
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38 
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■ »13 
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.09 


.12 


.15 ■ 


-.02 


.07 


.05 




.29 


.40 


.34 


.32 


.24 


i20 




' .34 


•.5i 


.37 


.60 


..52 


.11 




HA 
. UO 


.28 


.11 


.18 


.19 




.09 


.12 


.02 


.19 


.01 


.01 


.11 


.43 


.36 


.44 


.42 


.41 / 


.31 


■ .28 


-.06 


^ -.di 


.09 


ao 


.10 ' 


.11 


.09 


.42 


.-32 


.51 c 


.45 


.42 


.41 


.23 


.30- 


.22 


".39 . 


31 


.37 


.42 


.22 


.48 


.49 


.61 


'50 


.37 


.40 


.46 


.40 


.23 


.48. . 
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.50 


.53 


.24 


.39' 
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.35 


'.36 


.■29 
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.27 
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.28 


.39 ■ 


'.46 
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.39 


.35 


.53 


.43 
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.33 ■ 


.27 


.31 


• .34 


0 .47 


.40 


.40 


.48 


.28 



.07 
.26 

.32 
.36 

.39 
. 29 

.21 
-.06 

.27 
.44 



.03 
.28 

.29 
.36 

.41 
.32 

.18 
-.01 

.25 
.36 



.21 
.45 

.43 
.52 

.51 
.52- 

.35 
-.11 

.39 
.51 



.08 
. 36 

.36 
.42 

.48 
.38 

.19 
-.05 

.28 
.49 



-.op 

• .36' 

:.23 
f .49 

.-42 
I .37 

.25 

.25 
,.48. 



.09 
.42 

.32 
•.49 

.42 
.43 

• .24 
-.06 

.30 
..59 



*Jlating Scales 



-r.^)6 
'.■20 

'^33. 

.36 
.20 

.19 
-,-.04 

.24' 
.24 



^8 

.13 

.00.^ 

.30' 
.42 

.09 

*07» 

.35 
.07 ■ 

.37 
.30 

.51 
.44 

.31 
.32 

•38|* 
.40 

.11 
.33 

.32 
.46 

.44 
.36 

.21 
-.01 

• .33 
.48 



Accuracy 
Interest , , 

Learning Ability 
Job' Knowledge 



Rj = Dexterity \ i 
R^ = Low Need for Supervision 
R_ = Perseverance 
Rq! = Overall Rating 
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